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CMM IN A BOX 



Field of the Invention 

The present invention relates to a method for assisting and expediting an 
organization's progression through the levels of the Capability Maturity Model 
(CMM). Specifically, the present invention relates to a method and related system for 
arranging and administering an organization's infrastructure and a project of interest 
so that the organization and the product may be more mature, as measured by the 



The Capability Maturity Model® (CMM®) may refer specifically to the 
Capability Maturity Model for Software (S W-CMM) or, more generally, to a number 
of other process improvement models developed by the Software Engineering 
Institute (SEI) and registered to Carnegie Mellon University. The SW-CMM was the 
first model developed by the SEI, and it originally evolved from the need for the 
United States Department of Defense to have another measure besides "lowest 
bidder" in determining who should win project bids. Specifically, the Department of 
Defense desired a method to better compare and distinguish well designed and 
shoddy, defective products. The two major usages of the SW-CMM are: (1) as a 
model for Software Process Improvement (SPI) and (2) as a measure of the capability 
to produce quality systems. Specifically, the CMM may help a purchaser differentiate 
properly working product from an incomplete, nonfunctioning, poorly designed 
product by providing information on a producing organization and its production and 
development procedures. 

The CMM is an example of a model-based improvement approach that 
focuses on creation process quality. The rationale for this focus is that, unlike 
hardware, manufacturing software is essentially error free (i.e., the production of the 
disks containing the program), but the quality defects (i.e., bugs) are produced during 
the concept and development process. Therefore, wailing to identify defects aft^r, 
creation of the product is generally difficult and costly. The CMM may be used as a^ 
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guideline for prioritizing limited resources on the most important, foundational 
improvements. In the SW-CMM, Key Process Areas (KPAs) define "building 
blocks" based on industry best practices. The ultimate goal is to establish "continual 
improvement" of the software engineering process and the resulting products, kaizen 
(Statistical Process Control), which is common in nonsoftware engineering 
disciplines. The CMM is described more fully in Mark C. Paulk, The Capability 
Maturity Model: Guidelines for Improving the Software Process (The SEI Series) 
(Addison- Wesley Pub Co.) (1995). 

The Capability Maturity Model Integration— (CMMI—) was developed to 
integrate the SW-CMM and various other existing models into a common model. The 
developers of the CMMI are seeking to establish common terminology between the 
models, as well as identifying commonality and variability. The SEI is expected to 
release Version 1 . 1 of CMMI in the near future. 

The SW-CMM model defines five capability levels and identifies Key Process 
Areas (KPAs). The CMMI model replaces the KPAs with Process Areas (PAs). The 
lower levels of the CMMI and the related PAs focus mainly on management 
processes and industry minimal standards. Higher CMMI levels and the related PAs 
generally focus more on organizational and technical processes. The higher levels 
and their PAs also strive for "industry-best" practice. 

While the entire scope of the CMMI is vast and generally outside the range of 
this document, the various levels of the CMMI are now quickly described. At level 0 
or "Incomplete", a project has not yet started. Upon initiation and existence of the 
project, the project is at level 1. At "Initial" or level 1, the product conditions are ad 
hoc, chaotic, and high-risk. At "Repeatable" or level 2, the project may repeatedly 
perform some functions with difficulty. Relevant PAs to level 2 are Requirements 
Management (RM); Project Planning (PP); Project Monitoring and Control (PMC or 
PC); Supplier Agreement Management (SAM or SM); Process and Product Quality 
Assurance (PPQA or QA); Configuration Management (CM); and Measurement and 
Analysis (MA). 

At "Organizationally Defined" or level 3, the relevant PAs include 
Requirements Development (RD); Technical Solution (TS); Product Integration (PI); 
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Validation (Va); Verification (Ve); Organization Process Focus (OPF or PF); 
Organizational Process Definition (OPD or PD); Organizational Training (OT); 
Integrated Project Management (IPM or IM); Risk Management (RSKM or Rk); 
Decision Analysis and Resolution (DAR or DA); Organizational Environment for 
5 Integration (OI); and Integrated Teaming (IT). 

At "Quantitatively Managed" or level 4, the relevant PAs are Quantitative 
Process Management (QPM or QM) and Organizational Process Performance (OPP or 
OP). QPM relates to the informed and correct use of rigorous statistical techniques 
such as statistical process control (SPC), with the focus on removing specific or 

10 attributable causes of variance, and OPP relates to the use of statistical techniques to 

measure process efficiency. The fifth and highest level, "Optimizing", is basically 
equivalent to bottom-up process improvement or continuous improvement. In CMMI, 
the level 5 PAs are Organizational Innovation and Deployment (OID or ID) and 
Causal Analysis and Resolution (CAR or CA). 

15 The Capability Maturity Model for Software (SW-CMM) was the first, but not 

the only, model for improvement of software development. Some other models 
developed by the SEI include: Integrated Product Development CMM (EPD-CMM), 
which was renamed and incorporated into CMMI Integrated Product and Process 
Development (IPPD); People CMM (P-CMM) for Training, Career Development, and 

20 Human Resource-related issues; Personal Software Process 5 ™ (PSP sm ); Software 

Acquisition CMM® (SA-CMM); and Systems Engineering CMM® (SE-CMM), 
which is being incorporated into CMMI for Systems Engineering/Software 
Engineering. Similarly, FAA-iCMM (a model similar to CMMI and incorporating 
elements of SW-CMM, SE-CMM, and SA-CMM) was developed by the Federal 

25 Aviation Administration. 

Achieving higher levels of CMM maturity is a desirable goal in itself because 
it generally implies that an organization is producing a superior product and services 
since the higher levels of the CMM generally require the existence of infrastructure 
and procedures leading to better tested and developed software and other products. 

30 As suggested above, organizations also have secondary financial incentives to achieve 

higher CMM levels, because customers, such as the United States Department of 
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Defense, are increasingly requiring software suppliers to have a sufficiently high 
CMM level (e.g., at least level 2) before being awarded a contract. 

A threshold problem for many organizations is that the requirements for the 
different maturity levels are relatively complex to understand and implement. It is, 
5 therefore, a goal of the present invention to provide a method allowing businesses to 

achieve higher CMM levels without having to understand the complicated 
requirements of each level. 

Furthermore, the process of achieving higher CMM levels of increased 
maturity is typically a difficult, expensive, and time-intensive process. While some of 

10 the costs are unavoidable, many of the difficulties of achieving higher CMM levels 

occur because the requirements for the levels do not fit well within the general 
operations and structure of most organizations. Drastically changing an 
organization's structure and operations is generally a complex and difficult process. 
Therefore, another goal of the present invention is to provide a method that simplifies 

15 and potentially accelerates the process of modifying an organization's operations and 

structure to meet the requirements of the higher CMM levels. 

Similarly, many organizations also have difficulty implementing changes to 
achieve higher CMM or CMMI levels because the organization use of these maturity 
models as merely checklists of criteria. The maturity models, while serving as a 

20 measure to assess organizations, offer little guidance to organizations on 

implementation of the specified criteria. The random implementation of the items on 
a maturity model checklist results in increased time and cost for maturation in 
comparison to carrying out systemic changes that may concurrently satisfy multiple 
checklist items and assist the organization in achieving several checklist items. 

25 Furthermore, a piecemeal implementation of the CMM worsens the above-described 

problems of complexity and cost. It is, therefore, another goal of the present 
invention to provide a method by which organizations may implement systemic 
changes to achieve higher levels of CMM maturity. 

30 Summary of the Invention 
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In response to these and other needs, the present invention provides a method 
and related system for assisting and expediting an organization's transformation 
toward higher levels of the Capability Maturity Model (CMM) or other derivative 
maturity models. In particular, the present invention provides a method for producing 
a more mature product. A preferred embodiment of the method comprises the 
managing an organization developing the product, whereby the organizational 
management comprises managing personnel of the organization and implementing a 
product improvement process. The method may further comprise managing a project 
for developing the product and managing the delivery of the product. Furthermore, 
actions undertaken during the organizational management affects implementation of 
the project and delivery managements, and the actions undertaken during the project 
and delivery managements likewise affect implementation of the organizational 
management. In another embodiment, this method may be implemented using a 
combination of both electronic hardware and software. 

Brief Description of the Drawings 

A more complete understanding of the present invention and advantages 
thereof may be acquired by referring to the following description taken in conjunction 
with the accompanying drawings, in which like reference numbers indicate like 
features, and wherein: 

FIG. 1 is a flowchart depicting the steps in a method for producing more 
mature products in accordance with an embodiment of the present invention; 

FIGS. 2A-2J are flowcharts depicting the steps of the process stage of 
organization management in accordance with embodiments of the method of FIG. 1; 

FIGS. 3A-3D are flowcharts depicting the steps of the personnel stage of 
organization management in accordance with embodiments of the method of FIG. 1; 

FIGS. 4A-4F are flowcharts depicting the steps of program management in 
accordance with embodiments of the method of FIG. 1; 

FIGS. 5A-50 are flowcharts depicting the steps of project management in 
accordance with embodiments of the method of FIG. 1; 
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FIGS. 6A-6B are flowcharts depicting the steps of delivery management in 
accordance with embodiments of the method of FIG. 1; 

FIGS. 7A-7E are flowcharts depicting the steps of analysis stage of the 
delivery management of FIG. 6 A in accordance with embodiments of the method of 
FIG. 1; 

FIGS. 8A-8J are flowcharts depicting the steps of design stage of the delivery 
management of FIG. 6A in accordance with embodiments of the method of FIG. 1; 

FIGS. 9A-9M are flowcharts depicting the steps of build and test stage of the 
delivery management of FIG. 6A in accordance with embodiments of the method of 
FIG. 1; 

FIGS. 10A-10F are flowcharts depicting the steps of deployment stage of the 
delivery management of FIG. 6 A in accordance with embodiments of the method of 
FIG. 1; and 

FIGS. 1 1 A-B depicts systems for implementing the method of FIGS 1-1 OF in 
accordance with various embodiments of the present invention. 

Detailed Description of the Preferred Embodiment 
— As-generally^illustrate d in F I G. -frrtherpresent inveniion provides a CMMir^ 
BOX method 10 for easing and speeding an organization's transfortnatjon-toward 
higher levels of the above-described CMM hierarchy. Thgj(3MM in a BOX method 
10 generally comprises the steps of getting startpd^Torganization management 100, 
program management 400, project management 500, and delivery management 600. 
As suggested in FIG. 1, the CM^Hna BOX method 10 performs as a cycle in which 
actions performed during ^reorganization management 100 help control the current 
steps of program map^gement 400, project management 500, and delivery 
management 6pa Subsequently, the actions performed during program management 
400, projeefmanagement 500, and delivery management 600 adjust the step of 
organisation management 100. Each of these steps of CMM in a Box method 10 is 
d rinnrih e H in grenter detail below. 

IrUhese-disetisskrt^srit^houM^ 
i^MlV ^lfr^Rny method 1 0 preferably include_the xreation-or-updating -ofU^arious 
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documentation (or monuments) thaTdetSt^r^ ^e ri fy t h e fi x ee ution o f t asks p erformed, 
by the organization. These documents may be usedjQjiemenslTatecon^liance with 
the higher levels of the CMM or CM^£L--St>me of these documents are listed directly 
with the associated stepsjbtttlfcomplete listing is beyond the scope of the present 
application. A^sh6ft listing and summary of some of the various documents that may 
be cre#€cf or updated during the steps of the CMM in a Box method 10 is attached 
hereto as Appendix A. 

The-eMM4iuLiK>X^ 
20, the organization prepares to initiate the other steps inJhgjSMMTna BOX method 
10. In particular, the organization may review-tfi6*fequirements of the various 
management steps 100, 30Q^4e07and600. Similarly, the organization may review 
the CMM oiJ^MKlTarid their general requirements in order to better understand the 
gnats to hP L accompli s hed - during the va rious-S teps of the C M M in a Box method 1 0. 

Organization Management 

As illustrated in FIG. 1, Organizational Management 100 is divided into two 
stages, process step 200 and personnel step 300. The Organization management 
step 100 generally concerns activities related to the structure and activities of an 
organization. The process stage 200 contains the methodologies, process flows, tools, 
and templates to create and maintain a Software Engineering Process Group (SEPG). 
It should be noted that in the CMMI, the SEPG is replaced by a Process Group to 
allow for the inclusion of systems engineering. Thus, this application uses the SEPG 
to refer to a group overseeing software and non-software processes. As suggested by 
its title, the personnel stage 300 contains the methodologies, process flows, tools and 
templates to perform organizational design and development, measurement 
performance, and conduct organizational training. 

As depicted in FIG. 2 A, the process stage 200 consists of the steps of planning 
and organizing a SEPG, step 201; and of managing and improving the organization's 
processes, step 202. Step 201 is further subdivided into planning SEPG project 
execution (step 210) and organizing SEPG project resources (step 220). Likewise, 
managing and improving the organization's processes in step 202 may be subdivided 
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into controlling SEPG project work (step 230); rolling out and supporting SEPG 
projects (step 240), completing the SEPG project 290, and controlling process 
improvement (step 203). In turn, the step of controlling process improvement, 
step 203, consists of conducting a super SQA review, step 250; conducting 
5 assessments, step 260; conducting quarterly surveys, step 270; and conducting process 

improvements, step 280. 

In the planning and organizing of the SEPG in step 201, the organization first 
performs the planning of the SEPG project execution, step 210. While planning 
SEPG project execution in step 210, the SEPG defines its process improvement plan 
10 and subordinate plans for the fiscal year. Since the SEPG is a continuously operating 

project, plans are reviewed and updated annually, at a minimum, usually with the 
beginning of a new fiscal year. Step 210 begins at the initiation of the project to 
O define the pieces of an initial project plan and all subordinate plans that should be 

SJ used to manage the execution of the project. Using this information, the organization 

;ij 15 seeks to develop a SEPG project plan, a SEPG work plan, a communication and 

sponsorship plan, a configuration management plan, a risk management plan, and a 
training needs matrix, as these objects are defined in the CMM. The organization 
further performs decision analysis and resolution during the planning of the SEPG 
project execution, step 210. 
20 One-p€«siWe-processJbi^planning-the-&EPG-proj eefrexecutionrstep-gj^rig 

generally depicted in FIG. 2B. In an initial aspect ofthg^^laml^ project 
execution, step 210, the organization tml^is-tKeCMM in a BOX method 10 as 
needed. Specifically in step 2J K 2fffie organization determines whether to waive or 
skip steps in the CM^tiffa BOX method 10 as required by organization or the 
25 particular proje(5t For instance, the organization skip tasks that are inapplicable to a 

projecj^afid therefore unneeded to either achieving higher levels of maturity in the 

nr tn rWplnp rqnrR mature prn rinr.tfi, 

Another step in the SEPG project execution, step 210, is to develop a project 
plan, step 214. The project plan describes the project approach for the project 
30 timetable, metrics, organization, supplier agreement management, communication and 

sponsorship strategy, training, quality initiatives, software system development 
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process, configuration management, logistics, facilities, tools, and purchasing. It 
further describes the project approach for training, metrics tracking, and roles and 
responsibilities on the project. The organization may also use Decision Analysis and 
Resolution (DAR) to develop the Project Plan, as defined in the CMMI. 

The organization may further develop subordinate plans, step 216. The 
development of the appropriate subordinate plans, step 216, satisfies the needs of the 
project, such as the creation of subordinate plans for subcontractor management, risk 
management, communication and sponsorship, and configuration management, all of 
which are described in greater detail below. In the development of subordinate plans, 
step 216, the organization may further create a work plan. For instance, the 
organization may create a "bottom-up" or task- level project work plan based upon 
estimates where critical paths and dependencies are defined and managed within a 
project work-planning tool, such as Microsoft Project and Project Workbench®. 

Another aspect of the SEPG project execution process, step 210, is to develop 
project estimates, step 218. The organization may develop project estimates, 
step 218, using an estimating tool as a starting point for the estimates. For instance, 
estimates may be developed using the following steps: (1) tailor tasks and estimating 
model; (2) determine estimating factor values; (3) define work packages; 
(4) determine a timeline for the estimate; (5) reconcile a present estimate to an initial 
estimate; and (6) document assumptions used to form the estimates. The organization 
preferably further validates any estimates by verifying estimates against estimates or 
actual results from comparable projects. To form accurate estimates of available 
resources, the organization should further consider other resource-tapping activities 
such as community involvement, recruiting, mentoring, and training, when evaluating 
resources. 

Returning to FIG. 2A, the organization then continues the process stage 200 
and the planning and organizing the SEPG, step 201, by organizing the SEPG project 
resources, step 220. During step 220, the SEPG focuses on obtaining, assigning and 
training its human resources, and establishing the project's other physical resources 
including installation of tracking tools and document repositories. This task is 
performed iteratively as needed to organize, mobilize and manage SEPG resources 
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throughout the execution of the project. The organization performs step 220 as 
needed to organize the project's human resources, to establish other resources, to 
make work assignments and to any training needed to enable resources. Turning to 
FIG. 2C, the first step in organizing the SEPG project resources in step 220 is to 
refine resource needs, step 221. In this step 221, the organization defines the team 
organization structure, schedules the work, and defines the human and physical 
resource needs of the project. These tasks are performed in view of each project's 
requirements. By refining resource needs in step 221, the organization helps to ensure 
that project staffing and facilities needs are met on a timely basis without affecting the 
completion date and the quality of the work. The organization may complete this 
refining of resource needs in step 221 by: (1) determining project organization 
structure; (2) balancing a development schedule using human resource guidelines; and 
(3) refining physical resource needs that were outlined in the logistics, facilities, and 
tools section of the project plan formed in step 214. 

Returning to FIG. 2C, the organization continues the organization of the SEPG 
process resources in step 220 by establishing project standards and goals, step 222. 
The establishment of project standards and goals in step 222 is accomplished by 
developing, modifying, and adopting administrative and project- specific project 
standards and procedures. Examples of administrative procedures are employee 
availability checklists, time accounting procedures, status reporting, vacation 
scheduling, etc. Project standards and procedures include design and development 
standards, and the use of project specific tools. 

The organization continues the organizing the SEPG process resources in 
step 220 through organizing a project team in step 223, also illustrated in FIG. 2C. 
The selection of project team members is based on project requirements. Other 
elements in the organization of a project team are the finalization of the project team's 
organization structure and documentation in an organization chart in the project plan. 
The organization should further update the training needs matrix to document: (1) the 
training required of each project team member and (2) the proposed means for 
fulfilling the training. The training needs matrix is further used to track project team 
member training. In another implementation, organizing a project team in step 223 
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may further require the organization to determine, as a team, the project's mission, 
vision, and charter, and then to document these determinations in the project plan and 
orientation binder that are created as required to achieve higher maturity levels in the 



Returning to FIG. 2C, another task in the organization of SEPG project 
resources is to establish other resources indirectly needed for the SEPG project, 
step 224. Specifically, the organization performs this task by organizing the physical 
resources, such as hardware or software, provided by program management and 
developing the orientation and/or training needed to support the activities of the 
project team. The establishment of other resources in step 224 helps create a work 
environment that promotes communication, collaboration, and group cohesion. 

Also, as illustrated in FIG. 2C, the organization of SEPG project resources in 
process 220 further includes enabling resources, step 225. An organization performs 
this step 225 to orient and train team members, to coach and evaluate team members, 
and to manage the physical resources assigned to the project. The enabling of 
resources in step 225 aids the project manager in motivating and challenging team 
members, while helping to ensure that various project personnel believe their work to 
be important. Specifically, the organization should communicate the project's 
mission, vision, and charter to new team members. Large projects may also elect to 
formalize these items at the program level, and projects may conduct one or more 
meetings that include all team workers. 

Referring to FIG. 2A, another element in the process stage 200 is to manage 
and improve the organization's processes, step 202. The first step in the management 
and improvement of process is the control of SEPG project work in step 230. During 
the control of SEPG project work in the step 230, SEPG project management 
monitors the execution of the project against project plan and makes adjustments as 
necessary. Project Status Reports are prepared for the Project Sponsor. Potential and 
actual problems are identified through the measuring and monitoring of progress and 
performance against the SEPG Project Plan. Depending on the type of problem 
identified, an Issue, Risk, System Investigation Request (SIR) or Change Request 
(CR) is logged. The SIRs and the CRs are described in greater detail below. SEPG 
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Project management is expected to take appropriate corrective actions to resolve 
problems that are discovered. The controlling of SEPG project work in the step 230 is 
also illustrated in FIG. 2D and is now described in greater detail. The controlling of 
SEPG project work in the step 230 includes releasing work packages, step 23 1 . Work 
5 packages are generally described in the CMM criteria and generally relate to the tasks 

and functions given to the various workers in a project. To release work packages, 
the organization should (1) assemble and release work packages according to the 
work plan and (2) communicate the requirements of the work packages to the 
assigned team members. The project team then performs the work needed to develop 
1 0 the required deliverable good. During step 23 1 , the organization preferably acts to 

^ ensure that each team member understands assigned responsibilities, including target 

y dates and budgets. Furthermore, the organization should implement the project so 

O that each team member (1) is able to provide input regarding various responsibilities 

in 

and (2) accepts these responsibilities. 
Hj 15 As depicted in FIG. 2D, a following task in the control of SEPG project work, 

step 230, is measuring performance, step 232. The task of measuring performance in 
1=11 step 232 generally includes capturing actual results and calculation of metrics in order 

^ to manage performance. The capture metrics are outlined in the SEPG project plan 

□ formed in step 214 and include cost, effort, scope, quality, and schedule. The 

20 organization should further track project infrastructure and technical requirements, 

such as hardware, software, and performance requirements that were outlined during 
planning in step 210. The organization should also analyze any deviations from the 
project plan and identify, in a timely manner, the causes for the deviations. 

Concurrent with measuring of performance in the step 232 is managing 
25 performance, step 233, as illustrated in FIG. 2D. Managing performance in step 233 

generally requires the organization to manage project performance against the 
previously defined project and work plans. To manage project performance in view 
of the project and work plans, the organization proactively assesses performance, 
status, quality and risk. When the actual results from the development of the project 
30 do not match the plans, the organization should further determine alternative goals or 

actions. The implementing organization may further obtain approval for corrective 
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actions, and then take corrective actions. The corrective actions may include, but are 
not limited to, work process changes, team building, training, increased or decreased 
supervision, work assignment changes, reassignment of team members, initiation of 
risk responses, the change of requests to be pursued with program management as part 
of the configuration management process, project replanning changes that specify 
needed modifications to the project plan, project plan revisions (work package 
changes, etc.) or escalation to program management. The organization should also 
reevaluate project decisions throughout the project life cycle, when various project 
triggers or other issues, risks, etc. arise. The organization may also manage team 
member performance according to organizational and industry standards and tools. 

Continuing with FIG. 2D, following the measuring of performance in step 232 
and the managing of performance in step 233, the organization communicates project 
status, step 234. During step 234, the organization generally develops and 
communicates project status to all project stakeholders according to the project plan. 
The project stakeholders include project and senior management and other affected 
groups. The organization may further conduct status and review meetings involving 
affected groups as appropriate. During the communication of project status in 
step 234, the organization should document meeting minutes as required for the 
CMM. 

Continuing with FIG. 2D, following the communication of project status in 
step 234, the organization obtains acceptance of interim deliverable goods, step 235. 
Obtaining acceptance of interim deliverable goods in step 235 generally requires that 
the organization obtain acceptance of interim deliverables by all designated 
stakeholders, as appropriate, at key interim points throughout the project life cycle. 
Any acceptance of final deliverables takes place in connection with completing the 
program. 

Concurrent with the above-described steps 231-235, another task in the control 
of SEPG project work in step 230 is to execute project management processes, 
step 236. The organization should execute step 236 in conjunction with other project 
control activities, such as measurement activities and status reporting. Also, the 
project management processes may occur continuously, periodically, or may be event 
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driven. One project management process in step 236 is risk management, which 
addresses the identification, analysis, and avoidance/mitigation aspects of risk 
management on a project. During risk management, the organization may perform 
risk identification, during which the organization identifies, names, and describes the 
various risks. The organization should further generate a list of specific incremental 
risks in the project's risk tracking tool. For instance, the organization may document 
known triggers for a risk, the potential damage for each risk item, and references for 
the sources of risk. Another risk management task in step 236 is risk analysis, in 
which the organization analyzes the identified risks. In the risk analysis, the 
organization should classify the risks and include any additional information 
necessary to support the analysis. The organization may then select a rank/prioritized 
list of top risks. For instance, the organization may create a list of the top five risks to 
a project. Another risk management task is risk avoidance and mitigation. Risk 
avoidance activities address the sources of a risk, thereby reducing the probability that 
it would become a problem. For a top ranked or prioritized risk, the organization 
should identify how the risk can be avoided. Risk mitigation measures attack the 
consequences of a risk, reducing the risk's potential impact on the project. For the top 
ranked/prioritized risks, the organization may identify actions to reduce the impact of 
the risk if it occurs. The organization may also use Decision Analysis and Resolution 
(DAR) to assess the risks, where DAR is defined above. Many automated risk 
management applications are commonly available, and an organization may choose 
from these various risk management applications as needed to best fulfill the needs of 
the organization. 

Another task in the execution of project management in step 236 is scope 
management, which addresses the acceptance of requirements to define scope and the 
requirements to change control process. For instance, one scope management task is 
requirements development. During the task of requirements development, the 
organization identifies and documents requirements needed to promote and ensure 
bidirectional traceability, so that the organization may trace requirements between the 
development and the testing of the requirements. As with all work products, 
requirements are preferably placed under configuration management (CM), as defined 
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in the CMML Another scope management task is requirements acceptance, during 
which the organization documents and reviews requirements with all affected groups 
and obtains acceptance from the affected stakeholders. The organization should 
further establish baseline standards for satisfying the requirements. Another scope 
5 management task for the organization is making any required changes to the 

requirements and their baselines. The organization generally follows the project's 
change control process for any changes to baselined requirements. Namely, the 
organization submits a change request; reviews a change request; performs impact 
analysis, including cost, schedule and efforts impacts; determines disposition; 
10 implements change, including associated impact to other work products and activities; 

M and notifies requester and affected groups. Again, the organization may determine if 

S it is necessary to use DAR to assess changes in scope. 

!□ Another project management process in step 236 in the execution of the 

SJ project management processes is configuration management. This task addresses the 

Jij 1 5 set of activities performed to establish and maintain the integrity of the project work 

products throughout the project's life cycle. One set of configuration management 
ry tasks relates to configuration identification activities. During the configuration 

r\ identification activities, the organization identifies, names, and describes each of the 

O configuration items that should be placed under configuration management. In 

20 particular, all work products should be placed under some type of configuration 

management. During the configuration identification activities, the organization 
generally uses the CM plan to define a baseline for the configuration items and to 
indicate the level of configuration management for each item. 

Another configuration management process in step 236 is the configuration of 
25 control activities. Generally, the organization requests, evaluates, approves or 

disapproves, and implements changes to the baselined configuration items defined 
during the configuration identification activities. All of the configuration items 
should be archived and placed under the project's documented change control 
process. 

30 Configuration of status accounting activities is another configuration 

management process in step 236. During this process, the organization records and 
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reports the status of the project's configuration items. Similarly, the organization 
should further perform configuration audits. Specifically, the organization may, using 
the CM plan, determine the extent to which actual configuration items reflect the 
planned configuration items. The purpose of this task is to ensure that the entire 
configuration is correct and complete. The organization should further document 
results as required in the CMMI. 

Another project management process of the execution of the project 
management process in step 236 is issue management and escalation. This task 
involves the identification and documentation of issues using an issue tracking tool, as 
well as a review of the issue and an analysis of any impact on deliverables, scope, 
contingency, resources, costs, schedule, and/or quality. Specifically, the organization 
should identify a resolution approval party, an issue's owner, and determine expected 
time frames. The organization may also determine if it is necessary to use DAR to 
assess the issue, as described above. The organization may further research and 
identify issue solution alternatives. Subsequently, the organization may refer the 
issue to program/senior management when: (1) the project cannot resolve the issue 
internally, (2) when the issue impedes the progress of a project, and when the issue is 
beyond the authority of the project manager to resolve. These are generally issues 
that: (1) cannot be resolved within a project team, (2) are resolvable with action 
items, (3) can be escalated to the next level, (4) Are reactively discovered during the 
course of development, (5) affect program/project scope, costs, schedule, projected 
business performance, or high level design, (6) affect multiple projects or releases, 
and/or (7) involve groups outside the project that affect project delivery. The 
organization should accordingly monitor issues status and approve or reject 
resolutions. At the same time, the organization should communicate resolutions to 
stakeholders and affected parties and take corrective action as described above in the 
context related to management of performance tasks. 

Returning to FIG. 2D, another step during process of controlling the SEPG 
project work in step 230 is updating the project plan and subordinate plans, step 237. 
In particular, throughout the life cycle of the project, the project plan and subordinate 
plans (Risk Management, Configuration Management, Work Plan, Subcontractor 
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Management Plan, Community and Sponsorship Plan) should be updated as 
appropriate by the organization to reflect any changes on the project that would effect 
the content of the documentation. 

Referring again back to FIG. 2 A, another task of the management and 
improve process 202 in project stage 200 is the rollout and support of SEPG projects, 
step 240. During the rollout and support of SEPG projects in step 240, new projects 
to be supported by the SEPG are identified and SEPG processes and tools are 
delivered to them. SEPG Liaisons may conduct process reviews of the SEPG- 
supported projects. Other project-created items referenced during the rollout and 
support task include the Service Level Agreement, Tailoring & Waiver Request, 
Metrics Workbook and Metrics Plan. The organization performs this task of step 240 
to rollout SEPG processes and tools throughout the organization. The process of 
rollout and support of SEPG projects in step 240 is illustrated in greater detail in FIG. 
2E. Specifically, the rollout and support of SEPG projects in step 240 comprises the 
steps of identifying new projects, step 241; assigning a SEPG liaison, step 242; 
conducting a project kickoff, step 243; approving or disapproving waivers, step 244; 
collecting project metrics, step 245; conducting best practices reviews, step 246; 
reporting best practices status, step 247; and conducting project close out, step 248. 

Referring to FIG. 2E, during the identification of new projects in step 241, the 
organization should identify new projects that are in the planning stages. Then, in 
step 242, a SEPG rollout team leader assigns a SEPG liaison by evaluating the current 
workload among the available SEPG liaisons and select the most appropriate SEPG 
liaison for the current project. The rollout team leader preferably discusses the 
assignment with the SEPG liaison and sends a memo to the SEPG liaison informing 
him or her of the assignment, with a copy to the project manager and the SEPG 
program leader. 

Continuing with FIG. 2E, the next step of conducting a project kickoff in 
step 243 is conducting a kickoff meeting, preferably within 2 weeks of notification of 
support to be provided by SEPG. The SEPG liaison should schedule a time to meet 
with the project management team to discuss the kickoff. The SEPG liaison should 
also ask a project manager for project documentation such as the proposal, the 
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statement of work, estimating tool estimates, and the workplan, if available. This 
discussion is to establish the project organization, identify projects to support, and to 
ascertain the scope of the SEPG effort. 

Referring again to FIG. 2E, the next task in the rollout and support of SEPG 
projects in step 240 is to approve or disapprove waivers, step 244. In particular, the 
SEPG liaison or SEPG manager should work with the project or proactively identify 
any area where a project may need a waiver. A waiver request template should be 
available through the SEPG or through the SEPG liaison. A senior management 
official should sign the waiver request form, thereby acknowledging its risk and 
impact to the project. Also, the SEPG liaison should review the waiver request form 
for completeness and determine the disposition of the waiver request. The SEPG 
liaison then forwards the waiver request form to the SEPG project manager with a 
recommendation for disposition. Subsequently, the SEPG liaison informs the project 
manager of the disposition of the waiver request. 

Continuing with FIG. 2E, the next task is to collect project metrics, step 245. 
Step 245 may include one or more undertaking that help to ensure that the project 
metrics are collected in an organized and efficient manner. These undertakings may 
include collecting monthly project metrics, collecting best practice reviews, collecting 
quality review results, collecting stakeholder scorecard data, and collecting people 
satisfaction survey results. 

Continuing with FIG. 2E, another step is to conduct best practices reviews, 
step 246. With the conducting of best practice reviews, the SEPG liaisons should 
conduct monthly best practice reviews with project management in order to track and 
monitor compliance with CMMI requirements. The review criteria are based on the 
CMMI process areas and can be found within a best practices matrix. The reviews 
identify nonconformance items and areas for improvement. The SEPG liaisons 
should review the information gathered from the team and enter comments into the 
notes/comments section of the first best practice review matrix. During the meeting, 
the SEPG liaisons and project managers should review the matrix and determine 
which items have been met and those that would require additional information or 
documentation (artifacts). Based on the review, the SEPG liaisons should complete 
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the best practice matrix with documentation on additional information required from 
the project. Once the project reaches substantially complete compliance with the 
identified best practices, the best practice review focus becomes one of continued 
compliance and includes project team leaders and project team members. The SEPG 
liaisons should document and spot-check areas for compliance based on past reviews. 
These interviews may be conducted with or without project management in step 500, 
described below. 

Continuing with FIG. 2E, the next step is to report best practice status, 
step 247. There are two ways to report on the best practice status. The SEPG liaison 
may document Non-Conformance Items (NCI) and issues in a best practice 
notes/comments section and submit this to project management after the conclusion of 
the Best Practice review, preferably within two days. The project manager generally 
has a short time, such as one week, to provide a response for each NCI, including a 
target completion date for correcting the NCI. Alternatively, the SEPG liaison may 
complete a best practice "Dashboard" report with updated scores, open items, and 
risks. The report should be sent to project management and SEPG rollout and 
program leaders. 

Continuing with FIG. 2E, the next step is to conduct project close out, 
step 248. The organization may implement phase close out. In phase close out, the 
SEPG liaison may approve or disapprove the waivers, collect project metrics, conduct 
best practice reviews, and report on best practice status, etc. This process of rolling 
out and support of SEPG projects, along with the control of process improvements 
(step 203) described below, may then be repeated until the project is closed out. Next, 
in project closed out, the SEPG liaison works with the project manager and the 
management team to evaluate the overall impact and value of the SEPG program on 
the project. This evaluation should be done through the completion of a project close- 
out memo, verification of updates to the internal corporate resource by the project 
knowledge champion, verification of submission of the project's actual and estimated 
values to owners of the estimating tool via the profiling tool, collection of final 
project metrics, and collection of best practice and SEPG suggestions and comments. 
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Following the conducting of the close out in step 248, the organization may 
complete the SEPG projects in step 290, as depicted in FIG. 2J. To complete the 
SEPG projects in step 290, the SEPG Liaison reviews the Closing Memo and SQA 
Debrief created by projects that are complete and no longer require SEPG support. 
Specifically, the organization verify the completion of the supported projects, review 
the documentation produced in steps 230 and 240, and generate a list of best practices 
as desirable to produce a more mature product, respectively steps 292-296. 

Referring back to FIG. 2A, another task of the management and improvement 
process, step 202, in the project stage 200 is to control process improvements, 
step 203. The control of process improvements in step 203 brings together the tasks 
associated with controlling and conducting process improvement. As highlighted in 
the methodology, the Control Process Improvement, Rollout & Support SEPG 
Projects, and Complete SEPG Projects tasks are iterative in nature. Once processes 
and tools are improved, the SEPG is responsible for delivering these new processes 
and tools to its projects. Improving the control process in step 203 is comprised of the 
following steps: conducting a super software quality assurance (SQA) review 
(step 250), conducting mini-assessments and appraisals (step 260), conducting 
intermittent surveys (step 270) and conducting process improvements (step 280). 

One aspect of improving the control process in step 203 is to conduct a super 
SQA review, step 250. In step 250, the SEPG plans and organizes a Super Software 
Quality Assurance (SQA) review of its documents. A report is prepared based on the 
findings and reviewed with the SEPG Team. The results of this review help the 
SEPG to improve internal processes. The organization performs this step 250 to 
conduct software process and work product quality assurance reviews to verify 
project adherence to standards and procedures, such as any identified best practices. 
The quality program section of the Project Plan is described above in greater detail 
within the text accompanying FIG. 2B. Turning to FIG. 2F, the process of conducting 
a super SQA review in step 250 is described in greater detail. The first task of 
conducting the super SQA review in step 250 is to complete a SEPG project plan. In 
step 251, the process improvement (PI) team leader typically (1) identifies documents 
and processes to be reviewed; (2) ensures that documents in the Project Plan and 
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Work Plan are consistent; (3) identifies Super SQA reviewers, reviewees, and review 
criteria, (4) identifies roles and responsibilities; (5) identifies SQA metrics; (6) 
references the SQA plan in the quality section of the project plan; and (7) creates the 
SQA Plan. 

5 In the next step of conducting the super SQA review of step 250, the 

organization prepares for the super SQA review, step 252, as depicted in FIG. 2F. In 
one implementation of step 252, the PI team leader sets the super SQA review 
expectations. The PI team representative also submits reminder notifications to the 
super SQA reviewer based on any scheduled super SQA reviews, provides the Super 
10 SQA Reviewer with the Super SQA Reviewer training presentation and the SEPG 

u Program SQA Plan, and provides the super SQA reviewer with standards and 

y supporting documents to be reviewed. The PI team representative may further 

O provide the super SQA reviewer with document owner contact and availability 

ij information, as defined in the CMMI. The super SQA reviewer then typically gathers 

15 and reviews criteria/standards and supporting documents from the PI team 

s representative, reviews any super SQA reviewer training presentation, and schedules 

Py meetings with document owners. 

H Referring to FIG. 2F, the next step of conducting the super SQA review of 

Q step 250 is to conduct the super SQA review, step 253. The super SQA reviewer 

20 should review processes and documents against review criteria/standards, conduct 

interviews with document owners, identify nonconformance items, and follow up with 
the document owners as needed for meeting with the requirements of the desired 
CMM level. The document owner participates in the interview with the super SQA 
reviewer and remains available to answer questions. The PI Team Leader should also 
25 remain available to answer questions. 

In the next step 254, the organization, through the SQA reviewer, prepares an 
SQA report to document a detailed summary of findings and recommendations, as 
illustrated in FIG. 2F. Specifically, the SQA Report should include an item number, 
the date reported, and an accurate description of nonconformance items. The SQA 
30 reviewer may further distribute the SQA Report to the PI Team Leader, and schedule 

discussion of nonconformance items with the SEPG Program Lead. The PI team 
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representative also prepares and documents responses in the SQA Report, including 
an indication of whether the PI team representative agrees or disagrees with the 
reason statement, or otherwise determines the findings to be not applicable to the 
particular organization or project. 

Subsequently, the organization should discuss nonconformance items, 
step 255 in FIG. 2F. In step 255, the super SQA reviewer typically schedules and 
conducts a discussion of nonconformance items with the PI team leader, as well as 
verifying an adequate resolution of nonconformance items. Likewise, the PI team 
leader should discuss nonconformance items with the Super SQA Reviewer and refer 
disagreement items for facilitation to the SEPG Program Leader. A PI team 
representative should update the SQA report with proposed resolution(s) and 
projected completion date(s) for proposed changes/actions, and update and return the 
report, as well as all necessary documents, to the SQA reviewer for verification. The 
PI team representative further creates the System Investigation Requests (SIRs) and/or 
Change Requests (CRs) as necessary for the CMMI. The SIRs correspond to reports 
created to document errors in the product, and CRs conversely correspond to 
enhancements in the product that are beyond the scope of the original product. A 
SEPG liaison may then review and resolve escalated nonconformance items. 

Returning to FIG. 2F, the next step of conducting the super SQA review of 
step 250 is to track the super SQA metrics, step 256, by having the super SQA 
reviewer send the final report to the PI team leader. The PI team leader then forwards 
the final report and metrics to SEPG program leader, including metrics such as SQA 
schedule variance, and the number of nonconformance items. The PI team leader 
further tracks and reports on all open nonconformance items, while keeping project 
copies of documentation/reports. 

Returning to FIG. 2A, the next step in improving the control process in the 
step 203 is to conduct assessments, step 260. In step 260, the SEPG coordinates 
activities to determine the state of an organizations processes and practices. This 
assessment can take many forms and can range from informal process assessments, 
mini-appraisals or full-scale evaluations. In any of these situations the organization 
can utilize outside contractors to conduct the review. Step 260 generally includes 
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three stages: preparation, an on-site period, and wrap-up. After a series of interviews 
and review of documentation, the assessment results are then presented back to the 
organization. The organization should follow the same basic process when 
conducting an appraisal. In both cases, the organization may utilize an external group 
to execute the mini-appraisal and/or assessment. The process of conducting the mini- 
assessments and appraisals in step 260 is more fully illustrated in FIG. 2G. 

As depicted in FIG. 2G, the first task in the assessments in step 260 is to select 
projects, step 261. In step 261, the organization carefully selects projects used for the 
assessment in order to paint an accurate picture of the organization's processes. 
Generally, from one to four projects should be selected for assessment. Projects may 
be selected using the following criteria: (1) the project should be representative of the 
work (present and future) of the organization, and aligned with the business objectives 
of the organization; (2) the project should have at least six people working on it; 
(3) the project should have a duration of greater than three months; and (4) the project 
should not have a critical activity or milestone during the on-site period. 
Additionally, at least one of the projects should be in the build stage. Personnel from 
the selected projects should also be available for interviews and presentations. 

Returning to FIG. 2G, the next task in the mini-assessment and appraisal is to 
assess the development of an onsite schedule, step 262. The core of the assessment 
during step 260 is made up of the onsite period, which usually lasts from five to ten 
days. The onsite period consists of three basic activities: (1) gathering information 
through interview sessions with project leaders, team leaders, and functional area 
representatives; (2) mapping information to processes areas within the scope of the 
assessment through consolidation sessions; and (3) reporting findings and 
observations back to the organization through preliminary and final findings 
presentations. An executive session and a debriefing session is conducted to wrap up 
the on-site period. There is no limit to the number of hours spent on a particular 
activity; however, the assessment team is bound to the tasks that need to be completed 
before the next day. Training the assessment team is the other activity that can be 
considered part of the onsite period, as required, and can be scheduled just before the 
assessment activities begin. 
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As depicted in FIG. 2G, the next step in step 260 is to prepare assessment 
logistics, step 263. During step 263, a local assessment coordinator works with the 
assessment team leader to identify and prepare the logistics for conducting the on-site 
period. Logistical preparations include reservation of rooms for the on-site period 
(presentation, interview, and assessment team rooms); computer and presentation 
equipment (projectors, LAN connections, access to a phone, printer, copier, and 
general supplies); arranging for food and beverages, as well as accommodations for 
the assessment team, and confirming building/office access for the assessment team 
during the on-site period. 

The next step in step 260 is to select assessment participants, step 264, as 
depicted in FIG. 2G. In the selection of assessment participants in step 264, a good 
cross section of the organization must be considered when selecting assessment 
participants. This is done through interviewing individuals from each selected 
project, including the project leader, project team members, as well as personnel from 
supporting groups, such as quality assurance, configuration management, and/or the 
database group. Individuals who have been involved in developing or maintaining 
software in the organization also should be included in the list of interviewees. 
Participants selected for the assessment should come from different parts of the 
project life cycle, have at least six months of experience with the organization (and at 
least three months with the project), and be able to articulate their observations and 
opinions about the organization and its projects. Selected participants preferably can 
dedicate from six to eight hours to the assessment activities during the on-site period. 
For the assessment to take place, the assessment sponsor must be present for the 
initial and final presentations. 

Returning to FIG. 2G, the next step in the mini-assessment and appraisal is to 
develop organizational awareness of CMMI, step 265. The organization performs this 
task to train assessment participants on what the assessment will involve, how the 
assessment will be conducted, and what the organization expects of assessment 
participants. Awareness activities may include training sessions and distribution of 
awareness materials to everybody in the organization. The assessment sponsor, or the 
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organization's management (if different from the sponsor), must demonstrate their 
total support for the initiative. 

As depicted in FIG. 2G, the next step in the mini-assessment and appraisal is 
to collect process information, step 266. Step 266 is performed in preparation for the 
assessment of the collection of the documentation used in the current management 
and technical processes. Selected members of the organization fill out a maturity 
questionnaire to provide a baseline for scoping the assessment. The appropriate 
process documentation from both the organization and the projects being assessed 
should be collected to be reviewed by the team for the purpose of developing the 
assessment findings and observations. A documentation index should be created and, 
if required, the collected documents should be mapped to the CMMI process areas. 

Returning to FIG. 2G, the next step in step 260 is to conduct the assessment, 
step 267. In step 267, the assessment team visits the organization with the objective 
of mapping the organization's management and development processes against the 
CMMI. In particular, the assessment team should be trained. The assessment team 
should further conduct an opening meeting and interview project leaders, team 
leaders, and functional area representatives. The assessment team should further 
review collected documentation and consolidate information gathered and map it to 
process areas in the CMMI. Subsequently, the assessment team should conduct 
follow-up interviews, as required, and prepare and present preliminary findings to 
management and staff. Likewise, the assessment team should prepare and present 
final findings to the organization, incorporating feedback received from the 
preliminary findings presentation. The assessment team then conducts any executive 
and debrief sessions and prepares a final report. At the conclusion of the assessment, 
the assessment team files an assessment report with the Software Engineering 
Institute (SEI), including with the assessment the final presentation and the summary 
report. 

Returning to FIG. 2A, the next step in improving the control process in 
step 203 is to conduct a quarterly survey, step 270. The organization performs step 
270 to receive feedback from projects regarding the SEPG processes and tools. 
During step 270, the SEPG designs and delivers a quarterly Process Improvement 
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Survey to the projects it supports. The results of this survey are an input into the 
SEPG team's Process Improvement efforts. While named a quarterly survey, it 
should be appreciated that the survey may occur at other intervals and time periods. 
Results of this survey should be used to improve SEPG processes and tools. The 
5 process of conducting the quarterly survey in step 270 is depicted in FIG. 2H. The 

first task is to develop a survey process, in step 271, for administering the process 
improvement survey. This survey may be administered by the SEPG. The 
responsibilities for this task should be assigned to a sub-team. In developing the 
survey in step 271, the organization should consider the effect of the survey 
10 transmission medium and the methods through which the survey results will be 

analyzed. The organization should next develop the survey questions, step 272. The 
3 organization should select question on which the SEPG would like to receive 

3 feedback. Preferably, when developing survey questions, the organization should 

y choose nonleading questions. The organization should also preferably use a response 

g 15 scale that can be easily quantified, such as the Lickert scale. The organization should 

next, in step 273, administer the survey using the medium chosen in step 271. At this 
point, in step 274, the organization may evaluate and analyze the survey results 
received from respondents using the process developed in step 271. The organization 
may then use the survey results to improve the SEPG processes and tools, step 275. 
20 During step 275, the organization may also publicize the results of the survey. 

Returning to FIG. 2A, the next step in improving the control process in 
step 203 is to conduct process improvements, step 280. The organization performs 
step 280 to manage process improvement activities. During step 280, the SEPG takes 
the feedback it received from the SQA review, assessments, quarterly surveys, and 
25 feedback from other sources, and begins translating this feedback into improvements 

in SEPG processes and tools. Results from step 280 will be used to maintain, update, 
and develop SEPG processes, tools, and assets. Step 280 is illustrated in greater detail 
in FIG. 21. 

As depicted in FIG. 21, the first step in conducting process improvements is to 
30 maintain and improve processes and tools, step 281. In step 281, anyone in the 

organization and its external reviewers may identify a process improvement 
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opportunity (with a process, template, training, standard, tools, or the document 
repository itself). This could be in the form of an error (through a SIR), an 
improvement, an enhancement request (through a CR), or any other process 
improvement concern. The process improvement team leader should examine the 
process improvement opportunity, and a decision may be made on implementing the 
process improvement. If it is determined that the changes will be incorporated into 
the appropriate process asset (process, template, standard, training, etc.) in accordance 
with SEPG standards, then a SIR or CR may be documented to capture the change. 

Returning to FIG. 21, the next step in conducting process improvements, 
step 280, is to define and update processes and tools, step 282. In step 282, anyone in 
the organization may identify a new process or tool to be defined, documented, and/or 
built. The first type of process to be created is an internal SEPG process that uses a 
new process template to document process flows and descriptions. The second type 
of process to be created includes processes and tools that are part of the SEPG 
methodology. Process definition is submitted to a SEPG team leader for review and 
approval. If a process is approved, it will be scheduled for release to the organization 
and/or SEPG team, depending on the type of process submitted. 

As depicted in FIG. 21, the next step in conducting process improvements, 
step 280, is to pilot processes and tools, step 283. Once the process or tool to be 
piloted has been completed and approved, it is time to determine the pilot group, time 
frame, scope and functionality, roles and responsibilities, and entry and exit criteria, 
as part of step 283. The SEPG program manager may then work with a process asset 
owner to communicate the pilot's scope and expectations with the pilot group. The 
pilot group will be trained on the use and implementation of the process asset. The 
pilot is conducted with the process asset owner providing support to the pilot group in 
terms of providing clarification, additional training, or technical support, as necessary. 
At the end of the pilot period, the process asset owner debriefs with the pilot group or 
at least with a representative of the group to evaluate the pilot. Strengths and 
weaknesses of the process are identified, documented, and addressed. 

Returning to FIG. 21, the next step in conducting process improvements, 
step 280, is to roll out processes and tools, step 284. In step 284, once feedback from 
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the pilot group is incorporated into the process asset, it will be rolled out to the 
organization and/or SEPG team, as necessary. In step 284, the SEPG liaisons have 
the primary responsibility of communicating the new processes and tools to the 
organization's projects. 

5 Returning again to FIG. 21, the last step in conducting process improvements, 

step 280, is to assess and evaluate processes and tools, step 285. During step 285, the 
organization determines how processes and tools will be evaluated. The organization 
may further conduct intermittent or quarterly process improvement surveys. 



10 Personnel Stage 

Returning to FIG. 1 , a second process within the organization management 
step 100 relates to personnel management, step 300. The actions of step personnel 
hj management in step 300 generally relate to acquiring, organizing, and training the 

U 5 

SI organization's personnel as needed to encourage the development of more mature 

Jj 15 products and achieve higher levels of CMM maturity. Organizational Training of step 

!\ 300 is generally necessary to enable personnel to develop skills to meet specific roles 

HJ and responsibilities during solutions delivery in step 600 described below. The 

Q process of personnel management is generally depicted in FIG. 3 A and comprises the 

r: actions of designing a performance measurement infrastructure, step 310; executing 

|ssa 

20 organization design and development, step 320; and designing and deploying training, 

step 330; and is now briefly described. 

The designing of a performance measurement infrastructure in step 310 
generally relates to planing activities related to performance measurement to provide 
the organization with a means forjudging the effectiveness of the organization. The 

25 designing a performance measurement infrastructure in step 310 is summarized in 

FIG. 3B. The first step in step 310 is to validate and reach agreement on organization 
strategy, step 312. Step 312 generally involves the organization's key stakeholders in 
the development and/or validation of the organization's strategy, specifically the 
organization's mission, vision, and overall objectives. Because performance 

30 measurement cascades down from the organization strategy, the organization's 
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strategy should be understood and agreed upon by those accountable for 
implementing it. 

Returning to FIG. 3B, the next step in designing a performance measurement 
infrastructure in step 310 is to produce a performance measurement scorecard, 
step 314. In step 314, the organization uses a balanced scorecard to measure 
performance. The balanced scorecard is a measurement tool that translates strategic 
objectives into a coherent set of performance measures. The scorecard is "balanced" 
because it measures both leading and lagging indicators. These indicators are 
expressed in financial and nonfinancial terms. 

Subsequently, in step 316, the organization implements the scorecard, as 
depicted in FIG. 3B. Implementing the scorecard generally requires that the specific 
information for performance goals, metrics, and targets be collected from the front 
lines. Furthermore, the organization should compile at the strategic level each 
performance perspective, objective, metric, and target. Also, the organization should 
create and communicate top down, bottom up, and interactive performance measures. 
Subsequently, the organization should solicit feedback to test the effectiveness of 
metrics and how the performance measures fit in with the organization strategy. 

Turning to FIG. 3C, the next step in the personnel stage 300 is to execute the 
organization design and development, step 320. The organization performs step 320 
to plan activities related to organization design and development. Step 320 involves 
coordinating the tasks associated with defining a strategy for the organization, 
assessing the organization against this strategy, and deigning and implementing a new 
organization. Note that step 320 assumes that the organization has a Human Resource 
Organization with the skills to design and implement new elements of the 
organization. The organization should to have experience in organization design and 
development. The substeps of the organization of design and development in step 320 
are illustrated in FIG. 3C. 

The first task in step 320, as illustrated in FIG. 3C, is to identify an 
organization strategy, step 321. In step 321, business outcomes, core competencies 
and guiding principles are defined. These definitions will position the organization 
relative to business goals and objectives, vision and mission, management philosophy, 



-29- 



Atty. Dckt. N^^S 



769-006 




customer values, critical behaviors and competitive environment. Specifically, the 
organization should identify an organization strategy before detailed organization 
design. The organization should be designed to reflect not only where the company is 
relative to strategy, philosophy, and the value proposition of its customers, but also 
5 where it needs to achieve a competitive advantage in the future. The organization 

strategy sets the direction by defining business outcomes, core competencies, and 
guiding principles that will be used to anchor the organization design and 
development process. 

As depicted in FIG. 3C, the next step in the execution of the organization 
10 design and development in step 320 is to conduct an organization assessment, 

M step 322. It should be noted that the assessment differs from assessment used with 

q SEPG. The organization assessment helps to identify the supports and barriers to 

;2 transformation and build a case for implementation. An organization assessment in 

s| step 322 consists of assessing an organization's current situation, its future 

]|j 15 aspirations, and the gap between them; then identifying the initiatives required to fill 

these gaps. In step 322, enablers and barriers to organizational transformation are 
Hi identified and a case for implementation is built. This is accomplished through an 

assessment of the current organizational environment and future organizational 
aspirations, identifying the gaps between these two, and identifying a course of action 
20 to close those gaps. 

Referring to FIG. 3C, the next task in step 320 is to design an organization 
infrastructure, step 323, to create structures established to form individuals into the 
desired performing organization. The organization infrastructure's goal is to allow 
workers to effectively accomplish their tasks within the business process so that an 
25 overall goal is met. In step 323, the organization will design a competency model and 

design roles, jobs, teams and organizational structures. The competency model 
definition will document the knowledge, skills and other attributes/abilities associated 
with high performance on a job. The roles, jobs, teams and organizational structures 
will document the responsibilities associated with: the individual (roles), groups of 
30 related roles G°bs), groups of jobs (teams) and the span of control, reporting 

relationships and functional relationships of all of these components. Step 323 has 
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two subtasks - to design a competency model, step 324, and to design roles, jobs, 
teams and an organization structure, step 325. Steps 324-325 may be conducted 
iteratively and/or concurrently. In designing a competency model in step 324, the 
organization should group together related competencies to form a competency 
5 model. A competency is a cluster of related knowledge, skills, and other 

attributes/abilities associated with high performance on a job; and a competency 
model is a group of related competencies required to perform a career field such as 
team leader or technical coach. Similarly, during the process of designing an 
organization structure in step 325, the organization defines the roles played by 
10 individuals, the jobs they hold, the teams in which they work, and the relationship 

y, between teams. The organization should logically define roles for individuals on the 

p basis of their competencies, as decide in step 324. 

g Returning to FIG. 3C, the next task in step 320 is to verify and validate an 

•y organization structure, step 326. In step 326, all components of the newly defined 

;K 15 organizational infrastructure and reviewed to verify and validate that they meet the 

needs and goals of the organization. Specifically, the organization should verify and 
fij validate that any new organization design meets the needs of the business and is 

rl internally consistent. The organization should further confirm the new organization 

p design with any subject matter experts and initiative sponsors. Continuing with 

20 step 326, the organization should organize review sessions to validate how well the 

components of the new organization design (roles, jobs, teams, organization 

infrastructure, performance management infrastructure) fit together to support new 

initiatives. 

The next task in step 320, as illustrated in FIG. 3C, is to design a performance 
25 management infrastructure, step 327. In step 327, the organization's performance 

measurement scorecard is developed based on the organization's strategic objectives. 
This scorecard is then used to measure the organization's performance. Note that this 
task assumes that the organization has a Human Resource Organization with the skills 
to design and implement a performance measurement scorecard, and that the 
30 organization has experience in organizational performance management. Thus, in 

step 327, the organization defines a means for assessing, rewarding, and developing 
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the individuals in an organization. The performance management infrastructure has 
four components: (1) designing the performance management approach; (2) designing 
the performance appraisal instruments; (3) designing career progression; and 
(4) designing the compensation and reward structure. Overall, the organization 
5 should establish a system to reward individuals for desired contributions. 

The final task in step 320 is to determine an organization infrastructure 
mobilization approach, step 328, as illustrated in FIG. 3C. In step 328, the 
organization determines and mobilizes the resources required to staff the new 
organization infrastructure established in step 323. The organization may determine 
10 profiles for the ideal candidates, determine sizing and timing needs, and determine a 

sourcing approach. For instance, candidates may be profiled to fit job descriptions, 
the organizations new size may be determined and an approach to sourcing and 
staffing jobs may be finalized and executed.. 

^Returning to FIG. 3 A. the ne xLpracess in the per s onnel st .a ge . 30 .(Li s to de si* 
15 and deploy training, step 330. In step 33, the training needs of the organization ajs 

analyzed and a Training Plan is created, training is designed, developed anj^deliverer 
and post implementation support is provided. The organization performs step 330 to 
plan activities related to training employees. The design and^teployment of training 
during step 330 is illustrated in greater detail in FIG. 3D; As illustrated in FIG. 3D, 
the first task in step 330 is to conduct a training^r€eds analysis, step 331, during 
which the organization identifies, by nap*<the participants to be trained, along with 
the courses and modules on whiplrthese participants will be trained. In step 331, 
target audiences and participants are identified, and training courses and modules are 
planned. The traipklg needs analysis in step 331 may be conducted in two phases. 
25 During the fir^t phase, the organization gathers the high-level training needs for the 

organisation. Similarly, the second phase consists of gathering the detailed training 
-ful-llie oiganizalion. 
Returning to FIG. 3D, the next task in FIG. 3D is to develop a training plan, 
step 332, as needed, to describe the organization's overall training approach. In step 
30 332, the overall organizational approach to training is documented. The training plan 

formed in step 332 may include any of following sections/topics: Objectives; 
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assumptions; overall training approach; training courses, modules, and topics; training 
timeline; training logistics; and training evaluation. 

The next task in step 330, as illustrated in FIG. 3D, is to design training, 
step 333. In step 333, the training standards, templates, instructor and participant 
guides and the actual layout/format of training are developed. Specifically, the 
organization may develop the layout/format for the training materials. The 
development includes developing training development standards as well as templates 
for any instructor and participant guides. 

Similarly, the next task in step 330, as illustrated in FIG. 3D, is to develop 
training, step 334. In step 334, course content is created using the materials compiled 
during the training design step 333. The organization may implement step 334 by 
creating the course content using the training development standards and instructor 
and participant guides. Other material created in step 334 may include "Train-the- 
Trainer" materials, visuals, job aids/handouts, and tracking documents. Using the 
training materials developed during step 334, the organization may deliver training, 
step 335, such as a Train-the-Trainer session, a pilot training session, and the actual 
training. 

Returning to FIG. 3D, the next task in step 330 is to provide post- 
implementation support, step 336. In particular, during step 336, the organization 
should provide a short-term dedicated staff (e.g., one or two weeks) to support the 
users in applying what they've learned on the job. Furthermore, the support staff 
should be available to answer questions, identify and troubleshoot issues, and share 
best practices. 

THronghnt><^tep5^200 and 300, asj^eli^other-s teps in the C^IMJll^^aJU^ 
Method 10, the organization may need to commit to one^junere"acflons (not 
illustrated) as required to achieve highepxtatuTifyTevels in the CMM or the CMMI. 
Commit points are major depisit5nsregarding reporting the progress of present work 
and obtaining authorization to continue. Commit points define the boundaries of each 
stage aroyntfkey decisions related to content, context and course of action. For 
ii^mc^T^commirpoi nt may - be imp lemente d prio r loathe executing and desigrTPf-an^ 




-33- 



Atty. Dckt. N| 



769-006 




"Trrg^rizaticm-4ft&a structure in step 320, to require that the design of the new 

r or ganization stiuclu re^ust.be,apprav ed befor e furth erjj iiplementation can proc ogd^) 

Program Management 

Returning to tnj. i, a second primary-eemponent of the C1YTM injjJjQX 
method 10 of the present invention is program management stej^OOTProgram 
management step 400 generally concerns activities dire^tiyrelated to the creation and 
refinement of a program for implementing the C>0Cl in a BOX method 10. 
Specifically, program management 400 fopdses on the continuous oversight needed to 
support the delivery of a business spkmon through multiple projects and releases. 
Appropriate disciplines, techniqties, and tools are used in step 400 to plan and 
organize the work, and tp'fnanage the incremental delivery of the new business 
solution. As illusteafed in FIG. 4A, the program management stage 400 generally 
comprises the^feps of justifying the program (step 410); planning the program 
execution/(step 420); organizing program resources (step 430); controlling program 
work/(step 440); and completing the program (step 450). These individual steps are 
B^w-deserrbed in greater-detail — - 

As depicted in FIG. 4A, the organization may first justify the program, 
step 410. In step 410, a Program Business Case is prepared. The program business 
case approach is referenced to develop the business case. The business case is 
designed to secure stakeholder support for the program. Topics of the business case 
include the program's understanding of the current problem, the proposed solution to 
the problem that is to be implemented by the program, and a cost/benefit analysis. 
Justification of the program to all key stakeholders and sponsors helps in the 
successful execution, implementation and completion of the program. The program 
business case should provide economic justification for the change journey and for 
each program within the change journey. The program business case generally 
explains why the sponsoring organization should change, what value it receives by 
changing, and what steps are necessary for a successful change. The program business 
case addresses three main components, including business context and change 
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imperatives, value impact analysis, and change journey. The tasks in the justification 
of the program in step are generally illustrated in FIG. 4B. 

Referring to FIG. 4B, the organization first determines an economic evaluation 
approach, step 41 1, to obtain a "buy in" from the appropriate stakeholders in the 
sponsoring organization on the overall implementation approach for the program. 
Specifically, the organization tries to demonstrate the tangible benefits of a program 
to the affected parties. Step 411 attempts to show the process of implementing the 
program as an investment with positive, long-term benefits. 

Returning to FIG. 4B, the next task in step 410 is to create a model structure, 
step 412. In step 412, the organization obtains internal agreement regarding the 
structure of the model used to determine the benefits of implementing the program. 
For example, benefits to be derived may be expressed in terms of increased market 
share or reduced operating costs. In this way, affected parties may communicate the 
program's effects in terms of similar measures of costs and benefits. As suggested in 
FIG. 4B, the organization may also attempt to justify the program by forecasting 
baseline business performance, step 413. In other words, the organization may 
attempt to determine how the organization and its comprising units would perform 
without implementing the program. Continuing with FIG. 4B, another task in 
step 410 is to project net change journey benefits, step 414. The organization 
performs step 414 to predict and quantify the benefits that will be derived from 
implementing the program. 

The next step in step 410 is to assemble a business case, step 415, using the 
results assembled during steps 411-14. The organization may perform step 415 to 
document rationale for implementing the program. Ultimately, this documentation 
may serve as a motivational tool for change within the organization. 

Returning to FIG. 4A, the next task in the program management stage 400 is 
to plan the program execution, step 420. During step 420, the organization develops 
plans for the program itself, financial management and resource management. 
Program approaches are referenced during the creation of these documents. These 
plans guide the continued implementation of the program and are what the program 
will monitor itself against during later tasks. The individual tasks of step 420 are 
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illustrated in FIG. 4C. In step 420, the organization may develop a consolidated 
program plan, which documents the necessary tasks, effort, schedule, and costs for all 
releases of a business capability. The organization may also refine a program 
statement of work, and develop bottom-up project plans. Subsequently, the 
5 organization reconciles these plans with the top-down plans to generate a program 

baseline. The organization may have performed step 420 initially during program 
planning, in conjunction with or prior to the analysis stage 700 described below. 
Then, step 420 may be reinitiated during the course of the program as replanning is 
required by program management. 

10 Looking at FIG. 4C, the first task of the plan program execution in step 420 is 

to plan program processes, step 422. The organization may specifically determine all 
the management processes necessary to support the program. These relate to 
resources, vendors, quality, configuration, releases, issues, problems, risk, finances, 
contingency, and performance reporting. The organization may establish and 

15 document goals and metrics for each management process. The organization should 

begin this task package at the start of the program, and refine the management 
processes as the program progresses. The organization may perform this initial 
planning at the program level to help ensure that there are no gaps or overlaps of 
activities. While all the activities within the Delivering phase may be required for a 

20 particular business capability, it is unlikely that all of the activities should fall within 

the scope of a single project team. If the initial distribution of the activities to project 
type is done at the program level, the risk of missing or duplicate activities is limited. 

Returning to FIG. 4C, the next step in the plan program execution, step 420, is 
to develop a program budget, step 424. In step 424, the organization may establish a 

25 program budget that augments the cost baseline established in the program plan. The 

program budget provides the additional information needed by program management 
to manage the day-to-day financial affairs of the program. 

Another step in the plan program execution, step 420, is to develop a program 
communications plan, step 426, as illustrated in FIG. 4C. In step 426, the 

30 organization may identify and plan messages to program personnel, key program 



-36- 



I 



Atty. Dckt. N| 




769-006 




10 

Q 

o 

i — s 

U I 
'-■■3 

Jq 15 

1== 

20 
25 



executives, and other stakeholders in the program. In that way, step 426 addresses the 
communication needs within the program teams. 

Subsequently, the organization performs the task of finalizing the program 
plan, step 428, as depicted in FIG. 4C. In step 428, the organization may assemble 
the composite program plan. The Program Plan compiles the outputs from the plan 
management process 422 with the development of a program plan in step 424. The 
organization may then obtain executive and other appropriate management 
understanding and approval of the fully elaborated program plan and its components. 
The organization further briefs all key stakeholders (i.e., executive management, and 
impacted business operations) to ensure their understanding of, and commitment to, 
the program plan. This is crucial, because following this task: (1) the program should 
be described to the organization, and (2) more personnel should be assigned to the 
program. The organization may then take this opportunity to resolve any unclear or 
incorrect stakeholder expectations. 

Returning to FIG. 4A, the next step of the program management 400 is to 
organize program resources, step 430. During the step 430, resource requirements are 
analyzed and aligned so as to meet program objectives. As the program determines 
its resource needs, the Program Resource Request is completed to obtain the 
resources. Organize Program Resources is linked closely to planning the program's 
execution and pertains to staffing of the overall program. Under the Plan Program 
Execution task, the Program will plan for and deal with resource questions related to 
subordinate projects. Specifically, the organization may generally analyze resource 
requirements, initiate the procurement of goods and services, obtain human and 
physical resources from participating entities, assign these resources to projects, and 
release the resources upon project completion. The organization may perform 
step 430 throughout the life of the program created and implemented in step 400. 

As illustrated in FIG. 4D, in order to obtain and deploy resources, the 
organization may analyze resource requirements, step 432. This task analyzes 
resource requirements as defined in a program management resource plan. Resource 
requirements are consolidated from project needs and should generally include 
desired resource provider (generally the organization itself), if previously determined, 
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resource skill/type, and time period (such as monthly). Continuing with step 432, the 
organization may create a program resource management plan that forecasts resource 
needs by stage and capability release. 

Returning to FIG. 4D, the organization may further obtain and deploy human 
resources, step 434. Human resources are obtained by initiating a request with the 
Human Resources Organization, interviewing potential candidates, and selecting the 
candidate that best fits the requirements. Human resources are then assigned to 
projects as they arrive at the program. This task, alternatively, may be assigned to the 
projects. The program resource management plan may reflect actual information 
regarding the resource request. 

Returning to FIG. 4D, the organization may also procure and deploy physical 
resources, step 436. Physical resources are generally procured by initiating a resource 
request, evaluating the potential resources, and selecting the resource that best fits the 
requirements. Resources are then assigned to projects as they arrive at the program. 
The Physical Assets Inventory and the Program Resource Management Approach are 
generally both updated to reflect actual information regarding the resource request. 

Referring again to FIG. 4D, the organization also releases resources, step 438. 
When human resources are assigned to projects, they receive a "roll-off date 
indicating when these human resources are eligible for reassignment within or outside 
the program. If not reassigned inside the program, human resources are released to 
appropriate human resources departments for reassignment. Similarly, physical 
resource utilization is scheduled by each project, and these resources are returned 
upon completion of use. This process generally coincides with the completion of each 
stage of work. At that point, a determination should be made whether to retain or 
release the human and physical resources from the program. At this point in 
process 430, the entire process 430 may repeat if there are more program resources to 
organize, decision 439. 

FIGS. 4 A and 4E illustrate another step in the program management process, 
the control of program, step 440. In step 440, program management monitors 
program performance against program plans. Deviations from the plan are monitored. 
Corrective action is taken to resolve deviations as necessary. Program plans are 
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updated to reflect modifications to the program. Step 440 generally provides 
leadership to guide the planning and execution of program work. In step 440, the 
organization may maintain key working relationships within the program, while 
monitoring and developing the skills and performance of program management team 
5 members. The organization may further identify and assess problems with program 

performance, and specify corrective actions as needed. The organization may 
evaluate program metrics to determine progress toward program objectives, and to 
determine whether or not the current metrics are still relevant. The organization may 
further assess whether or not the program is on track by reviewing program, project, 
10 and vendor performance. 

M" The first task of step 440 is to administer the program, step 441 as illustrated 

g - in FIG. 4E. An effective program administration results in a planned, organized, and 

{■jj managed program management office performing a wide range of cost-effective 

y i 

SJ activities. As required, the teamwork environment requirements list deliverable 

Jq 15 should be updated to reflect relevant changes in the program. Program leaders should 

also strive to maintain a culture that encourages program participants to achieve 
rLJ maximum results. Program leaders should also communicate the common program 

%Tj vision to inspire others to support program goals. 

p A second task in step 440 is to report performance, step 442, as illustrated in 

20 FIG. 4E. The organization may process and prepare reports for cost/schedule and 

other performance data (e.g., quality, risk, resource, etc.). This should involve a 
standard set of reports as defined in the program performance reporting approach 
section of the program plan. Any ad hoc reports requested by program management 
may also be prepared. 
25 Returning to FIG. 4E, another task in step 440 is to perform financial 

management, step 443. Specifically, the organization may report, monitor, and 
account for the program's financial performance and results by performing the 
financial management functions as specified in the Program Financial Management 
Approach section of the Program Plan. Similarly, in step 444, the maintenance of 
30 administrative policies and standards, the organization may update and refine the 

administrative policies and standards on the basis of their effectiveness and the 
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evolving needs of the program, as illustrated in FIG. 4E. The organization should 
further communicate the changes to the program team members. 

Returning to FIG. 4E, the next task in step 440 is to conduct, as necessary, 
ongoing program orientation and training, step 445. In step 445, the organization may 
5 conduct periodic orientation and training sessions as new members join the program, 

as new types of training are required, and as team members need additional career 
development opportunities. Likewise, in step 446, the organization monitors a 
program communications plan to help to ensure that the appropriate groups 
accomplish their responsibilities. The program management office itself may also be 
10 responsible for performing some of the activities as directed in the program 

j«£ communications plan. 

:Sj In another group of steps illustrated in FIG. 4F, the organization may complete 

Q the program, step 450. In step 450, a program closeout report is prepared along with 

y other program closeout documentation. The program is demobilized and 

'%l 1 5 responsibility for the program is transferred to the necessary parties. The organization 

achieves an orderly and successful program closure by formally transferring 

|sss 

iy responsibility for the solution components to the operational units, obtaining formal 

r\ management acceptance of the competitive solutions delivered, releasing the 

p remaining human and physical resources to their providing organizations/owners, and 

20 completing a disposition of all program documentation and other materials. 

As illustrated in FIG. 4F, one step in the completing the product is to complete 
documentation, step 452. The activities needed to complete all program 
documentation include preparing any final documentation needed to close the 
program, including final cost, performance reports, etc. Additionally, a final review 
25 of the documentation is performed in step 452 to ensure that it is complete and 

conforms to program standards. The organization should also identify materials that 
should be shared across the organization, especially process improvements, 
methodologies, techniques, estimating models, and reusable components. The 
organization should also take steps to ensure that the materials are included in the 
30 appropriate repositories. The program documentation and other materials are 

transferred to any appropriate locations. Key deliverables are sent to the software 
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engineering process group team, as determined. A summary of the program's final 
disposition, assets, records, and other appropriate, relevant information should be 
contained in the program closeout report deliverable. 

Continuing with FIG. 4F, another step in the completion of the product is to 
transfer program responsibility, step 454. This activity transfers responsibility for the 
business capability to the appropriate organizational unit(s). Responsibility is 
assumed by the organizational units responsible for the continuing operation, 
maintenance, and use of the business capability and its underlying components. 

Returning to FIG. 4F, another step in the completion of the product is to 
demobilize the program, step 456. The resources to be released include the remaining 
program participants and all facilities (including furniture and equipment). The human 
resources are returned to the organizational units that provided them. The physical 
resources are released or returned to their owners. Any remaining procurement 
agreements (purchase orders, contracts, leases, rental agreements, etc.) are closed out. 

Project Management 

Returning to F iG^JhoJ^MMr m a BOX method 10 gen e rally callo fortt^e - 
organizations to concurrently perform project management 500 with the^rogram 
management 400. The project management 500 is generally depicted^n FIGS. 5A- 
50. Project management 500 generally concerns activities an^smictures directly 
related to the creation and refinement of a project or product for sale. Project 
management 500 controls the delivery of the specifip^omponents from which a 
business solution is derived through the balanced management of Scope, Quality, 
Effort, Risk and Timeline (SQERT). Project management 500 focuses on making 
critical decisions and managing risj^hat will ensure the delivery of the promised 
scope, on time and within budget at the agreed-upon levels of quality. When a 
program management fiiriction exists, project management works closely with 
program management to execute the SQERT activities in relation to the delivery of 
multiple proj^ts under one overall program. As illustrated in FIG. 5 A, project 
managefnent 500 generally includes planning of project execution (step 510); 
org anization of proj e ct r es ouro es-(step-52Q)- , c oi iUul piuject work- (st e p i ^O)^ 
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^mplpfinn nf thp prnjnrt (rtep ^/iO\ an S OA review - execution f-£tpp ^50V and 
su ppli e r agreement management (step 560\ 

FIG. 5B presents the individual tasks required in the planning of project 
execution, step 510. The organization may perform this task package 510 at project 
initiation to define pieces of the initial Project Plan and subordinate plans that should 
be used to manage the execution of the project. The tasks associated with Plan 
Project Execution, such as planning and estimating, are performed throughout the 
project lifecycle at predefined decision points, and whenever replanning is required. 
During the planning of project execution in step 510, the organization may tailor the 
process, step 512, to suit a project's needs by using known tools or means. The 
organization may further request a waiver for any required steps that should not be 
followed on the particular project. 

The organization may further implement the planning of project execution, 
step 510 through the development of a project plan, step 514, as illustrated in 
FIG. 5B. The organization may perform this task using a template to customize a 
specific project. The project plan describes the project approach for the project 
timetable, metrics, organization, supplier agreement management, communication and 
sponsorship strategy, training, quality initiatives, software system development 
process, configuration management, logistics, facilities, tools, and purchasing. The 
project plan also describes the project approach for training, metrics tracking, and 
roles and responsibilities on the project. The organization may further use a best 
practices matrix, a metrics plan, a DAR reference document, and a training needs 
matrix to develop the project plan, as defined in the CMMI. The DAR reference - 
document describes the formal DAR process and provides guidelines for identifying 
DAR triggers, setting thresholds, and selecting the best techniques. This information 
should be used to complete the quality program section of the project plan. The 
metrics plan generally contains the list of required and recommended metrics that a 
project should include in the project plan. 

The planning of project execution, step 510, continues in FIG. 5B with the 
development of subordinate plans, step 516. In step 516, the organization may 
develop the appropriate subordinate plans to satisfy the needs of the project. For 
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instance, the organization may define, as needed, subordinate plans for subcontractor 
management, risk management, communication and sponsorship, and configuration 
management. All projects require the creation of a work plan, and an organization 
may create a bottom-up or task-level project work plan based upon estimates. Critical 
5 paths and dependencies are defined and managed within the project work-planning 

tool, such as the Microsoft Project and Project Workbench®. 

Returning to FIG. 5B, the next step in plan project execution 510 is to develop 
project estimates, step 518. The development of project estimates in step 518 is 
analogous to the development of project estimates in step 218, as described above in 
10 FIG. 2B. Specifically, the organization may develop project estimates, step 518, 

y* using an estimating tool as a starting point for the estimates. For instance, estimates 

q may be developed using the following steps: (1) tailor tasks and estimating model; 

hj (2) determine estimating factor values; (3) define work packages; (4) determine a 

S| timeline for the estimate; (5) reconcile a present estimate to an initial estimate; and 

15 (6) document assumptions used to form the estimates. The organization preferably 

further validates any estimates by verifying estimates against estimates or actual 
ill results from comparable projects. To form accurate estimates of available resources, 

n the organization should further consider other resource-tapping activities, such as 

O community involvement, recruiting, mentoring, and training, when evaluating 

20 resources. 

Another step of the project management 500 is to organize project resources, 
step 520, as illustrated in FIG. 5C. The organizing of project resources in step 520, as 
well as in substeps 521-25, are analogous to steps 220-25, described above in FIG. 
2C. The organization can perform these tasks as needed to organize the project's 
25 human resources, establish other resources, to make work assignments, and to develop 

training enabling resources. In step 520, the project focuses on obtaining, assigning 
and training its human resources and establishing the project's other resources. This 
task is performed iteratively as needed to organize, mobilize and manage project 
resources throughout the execution of the project. 
30 Turning to FIG. 5C, the first step in organizing the project resources in 

step 520 is to refine resource needs, step 521 . In this step 521, the organization 
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defines the team organization structure, schedules the work, and defines the human 
and physical resource needs of the project. These tasks are performed in view of each 
project's requirements. By refining resource needs in step 521, the organization helps 
to ensure that project staffing and facilities needs are met on a timely basis without 
affecting the completion date and the quality of the work. The organization may 
complete this refining of resource needs in step 521 by (1) determining project 
organization structure; (2) balancing a development schedule using human resource 
guidelines; and (3) refining physical resource needs that were outlined in the logistics, 
facilities, and tools section of the project plan formed in step 210. 

Returning to FIG. 5C, the organization continues the organization of the 
process resources in step 520 by establishing project standards and goals, step 522. 
The establishment of project standards and goals in step 522 is accomplished by 
developing, modifying, and adopting administrative and project- specific project 
standards and procedures. Examples of administrative procedures are employee 
availability checklists, time accounting procedures, status reporting, vacation 
scheduling, etc. Project standards and procedures include design and development 
standards, and the use of project- specific tools. The establishment of these standards 
and procedures preferably improves the organization's communication, operating 
efficiency, and overall control of the project. 

The organization continues the step of organizing the process resources, 
step 520, through organizing a project team in step 523, as also illustrated in FIG. 5C. 
The selection of project team members is based on project requirements. Other 
elements in the organization of a project team are the finalization of the project team's 
organization structure and documentation in the organization chart of the project plan. 
The organization should further update a training needs matrix to document (1) the 
training required of each project team member and (2) the proposed means for 
fulfilling the training. This document is used to track project team member training. 
In another implementation, organizing a project team in step 523 further requires the 
organization to determine, as a team, the project's mission, vision, and charter, and 
then to document these determinations in a project plan and orientation binder that is 
created as required for the CMM. 
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Returning to FIG. 5C, another task in the organization of project resources in 
step 520 is to establish other resources, step 524. Specifically, the organization 
performs this task to organize the physical resources, such as hardware or software, 
provided by program management and to develop the orientation and/or training 
5 needed to support the activities of the project team. The establishment of other 

resources in step 524 helps create a work environment that promotes communication, 
collaboration, and group cohesion. 

Also as illustrated in FIG. 5C, the organization of project resources in process 
520 further includes enabling resources, step 525. Organizations perform this step to 
10 orient and train team members, manage the physical resources assigned to the project, 

and coach and evaluate team members. The enabling of resources in step 525 aids the 
jjjjjjj project manager in motivating and challenging team members and while helping to 

ensure that various project personnel believe their work to be important. Specifically, 
N! the organization should communicate the project's mission, vision, and charter to new 

Jrj 1 5 team members. Large projects may also elect to formalize these items at the program 

level, and projects may conduct one or more meetings that include all team workers. 
Hi As illustrated in FIG. 5D, another step in the project management 500 is to 

control project work, step 530. In step 530, project management monitors the 
execution of the project against project plans and makes adjustments as necessary. 
20 Project Status Reports are prepared for the Project Sponsor. Potential and actual 

problems are identified through the measuring and monitoring of progress and 
performance against the Project Plan. Depending on the type of problem identified, 
an Issue, Risk, SIR or CR is logged. Project management is expected to take 
appropriate corrective actions to resolve problems that are discovered. Step 530 and 
25 constituting tasks 531-37 closely correlate, respectively, to steps 230-37, described in 

FIG. 2D and its accompanying text. The organization performs step 530 to control 
project execution throughout the project's life cycle. The control of project work in 
step 530 includes identifying potential and actual problems by monitoring and 
measuring progress against the project plan. 
30 As illustrated in FIG. 5D, the controlling of project work in step 530 begins 

with the releasing of work packages, step 531. To release work packages, the 
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organization should assemble and release work packages according to the work plan, 
and communicate their requirements to the assigned team members. Work packages 
are generally described in the CMM criteria and generally relate to the task and 
functions given to the various workers in a project. The project team then performs 
5 the work needed to develop the required deliverable good. During step 531, the 

organization preferably acts to ensure that each team member understands assigned 
responsibilities, including target dates and budgets. Furthermore, the organization 
should encourage each team member to provide input regarding various assigned 
responsibilities, including target dates and budgets, and to accept and carry out these 
10 assigned responsibilities. 

H= As depicted in FIG. 5D, a following step in the control project work, step 530, 

q is measuring performance, step 532. The measuring of performance in step 532 

y generally includes capturing actual results and calculation of metrics in order to 

S| manage performance. Capture metrics, as outlined in the organization metrics plan 

J-j 15 formed in step 510, include cost, effort, scope, quality, and schedule. The 

organization should further track project infrastructure/ technical requirements, such 
ITJ as hardware, software, and performance requirements, that were outlined during 

| 5 

J] planning in step 510. The organization should also analyze any deviations from the 

project plan and identify, in a timely manner, the causes for the deviations. 

20 Concurrent with the measuring of performance in step 532 is managing 

performance, step 533, as illustrated in FIG. 5D. Managing performance in step 533 
generally requires the organization to manage project performance against the 
previously defined project and work plans. To project performance in view of the 
project and work plans, the organization proactively assesses performance, status, 

25 quality, and risk. When the actual results from the development of the project do not 

match the plans, the organization should further determine alternative goals or 
actions. The implementing organization may further obtain approval for corrective 
actions, and then take corrective actions. The corrective actions may include, but are 
not limited to, work process changes, team building, training, increased or decreased 

30 supervision, work assignment changes, reassignment of team members, initiation of 

risk responses, the change of requests to be pursued with program management as part 
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of the configuration management process, project replanning changes that specify 
needed modifications to the project plan, project plan revisions (work package 
changes, etc.) or escalation to program management. The organization should also 
reevaluate project decisions throughout the project life cycle, when project triggers or 
5 other issues, risks, etc. arise. In step 533, the organization may also manage team 

member performance according to organizational standards and tools. 

Continuing with FIG. 5D, following the measuring of performance in step 532 
and the managing of performance in step 533, the organization communicates project 
status, step 534. During the step 534, the organization generally develops and 
10 communicates project status to all project stakeholders according to the Project Plan. 

■* The project stakeholders include project and senior management and other affected 

=~ 

5 groups. The organization further conducts status and review meetings involving 

~ 2 affected groups as appropriate. During the communication of project status in 

-J step 534, the organization should document meeting minutes as required for the 

q 15 CMM. 

Continuing with FIG. 5D, following the communication of project status in 
U step 534, the organization obtains acceptance of interim deliverable goods, step 535. 

Obtaining acceptance of interim deliverable goods in step 535 generally requires that 
the organization obtain acceptance of interim deliverables by all designated 
20 stakeholders, as appropriate, at key interim points throughout the project life cycle. 

Any acceptance of final deliverables takes place in connection with completing the 
program. 

Another task in the control of project work in step 530 is to execute project 
management processes, step 536. The organization should execute these processes in 
25 conjunction with other project control activities, such as measurement activities and 

status reporting. Also, the project management processes may occur continuously, 
periodically, or may be event driven. One project management process in step 536 is 
risk management, which addresses the identification, analysis, and avoidance/ 
mitigation aspects of risk management on a project. One project management process 
30 is risk identification, during which the organization identifies, names, and describes 

the various risks. The organization should further generate a list of specific 
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incremental risks in the project's risk-tracking tool. The organization documents 
known triggers for a risk, the potential damage for each risk item, and references for 
the sources of risk. Another risk management task in step 536 is risk analysis, in 
which the organization analyzes the identified risks. In risk analysis, the organization 
should classify the risks and include any additional information necessary to support 
the analysis. The organization may then select a rank/prioritized list of top risks. For 
instance, the organization may create a list of the top five risks to a project. Another 
risk management task is risk avoidance and mitigation. Risk avoidance activities 
address the sources of a risk, thereby reducing the probability that it should become a 
problem. For a top-ranked/prioritized risk, the organization should identify how the 
risk can be avoided. Risk mitigation measures attack the consequences of a risk, 
reducing the risk's potential impact on the project. For the top-ranked/prioritized 
risks, the organization may identify actions to reduce the impact of the risk if it 
occurs. The organization may also use DAR to assess the risks. 

Another project management process in the execution of project management 
processes in step 536 is scope management, which addresses the acceptance of 
requirements to define scope and the requirements of change control process. For 
instance, one scope management task is requirements development. During the task 
of requirements development, the organization identifies and documents requirements 
needed to promote and ensure bi-directional traceability, so that the organization may 
trace requirements between the development and the testing of the requirements. As 
with all work products, requirements are preferably placed under configuration 
management (CM), as defined in the CMMI. Another scope management task is 
requirements acceptance, during which the organization documents and reviews 
requirements with all affected groups and obtains acceptance from the affected 
stakeholders. The organization should further establish baseline standards for 
satisfying the requirements. Another scope management task for the organization is 
to make any required changes to the requirements and their baselines. The 
organization generally follows the project's change control process for any changes to 
baselined requirements. Specifically, the organization submits a change request; 
reviews a change request; performs impact analysis, including cost, schedule, and 
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efforts impacts; determines disposition; implements change, including associated 
impact to other work products and activities; and notifies requester and affected 
groups. Again, the organization may determine if it is necessary to use DAR to assess 
changes in scope. 

5 Another project management process in step 536 in the execution of the 

project management processes is configuration management. This task addresses the 
set of activities performed to establish and maintain the integrity of the project work 
products throughout the project's life cycle. One set of configuration management 
tasks relates to configuration identification activities. During the configuration 
10 identification activities, the organization identifies, names, and describes each of the 

M configuration items that should be placed under configuration management. In 

Q 

q particular, all work products should be placed under some type of configuration 

y management. During the configuration identification activities, the organization 

y § 

SI generally uses the CM plan to define a baseline for the configuration items and to 

in 

fe g 15 indicate the level of configuration management for each item. Another configuration 

management process in step 536 is configuration of control activities. Generally, the 
W organization requests, evaluates, approves or disapproves, and implements changes to 

^ I the baselined configuration items defined during the configuration identification 

activities. All of the configuration items should be archived and placed under the 
20 project's documented change control process. Configuration of status accounting 

activities is another configuration management process in step 536. During this 
process, the organization records and reports the status of the project's configuration 
items using a configuration management status report. Similarly, the organization 
should further perform configuration audits. Specifically, the organization may, using 
25 the CM plan, determine the extent to which actual configuration items reflect the 

planned configuration items. The purpose of this task is to ensure that the entire 
configuration is correct and complete. The organization should further document 
results as required in the CMMI, using a configuration audit. 

Another project management process of the execution of the project 
30 management process in step 536 is issue management and escalation. This task 

involves the identification and documentation of issues using an issue tracking tool, as 
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well as a review of the issue and an analysis of any impact on deliverables, scope, 
contingency, resources, costs, schedule, and/or quality. Specifically, the organization 
should identify a resolution approval party, an issue's owner, and determine expected 
time frames. The organization may also determine if it is necessary to use DAR to 
5 assess the issue, as described above. The organization may further research and 

identify issue solution alternatives. Subsequently, the organization may refer the 
issue to program/ senior management when: (1) the project manager cannot resolve 
the issue internally, (2) when the issue impedes the progress of a project, and when 
the issue is beyond the authority of the project manager to resolve. These are 
10 generally issues that (1) cannot be resolved within a project team, (2) are resolvable 

M with action items, (3) can be escalated to the next level, (4) are reactively discovered 

Q 

during the course of development, (5) affect program/project scope, costs, schedule, 
D projected business performance, or high-level design, (6) affect multiple projects or 

SJ releases, and/or (7) involve groups outside the project that affect project delivery. 

*M 15 The organization should accordingly monitor issues status while approving or 

* rejecting resolutions. At the same time, the organization should communicate 

ly resolutions to stakeholders and affected parties and take corrective action as described 

" above in the context related to management of performance tasks. 

O Returning to FIG. 5D, another step during the process of controlling the 

20 project work in step 530 is to update the project plan and subordinate plans, step 537. 

In particular, throughout the life cycle of the project, the project plan and subordinate 
plans (Risk Management, Configuration Management, Work Plan, Subcontractor 
Management Plan, Community, and Sponsorship Plan) should be updated as 
appropriate, by the organization to reflect any changes on the project that should 
25 affect the content of the documentation. 

Another step of project management 500 is to complete the project, step 540, 
as illustrated in FIG. 5E. In step 540, project closeout is performed and overall 
project results are evaluated. Project Management verifies that all activities for a 
project are complete so that all resources can be released and all documentation and 
30 responsibilities can be transferred to the necessary parties. In this way, step 540 

enables Project Management and the Project Sponsor to measure the success of the 
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project and use results of the project as inputs to future efforts. A first step in 
completing the project in step 540 is to obtain a formal acceptance of deliverable(s), 
step 541, and this task 541 entails obtaining sign-offs on the final deliverables from 
the appropriate stakeholders. In effect, each stakeholder should agree that the project 
5 is, in fact, complete. Another step in completing the project during step 540 is the 

preparation of final documentation, step 542, in which the organization completes 
final revisions and packaging of deliverables. Likewise, the organization furthers the 
completion of the project in step 540 by transferring responsibility for deliverables, 
step 543, to formally transition responsibility for the deliverables to the appropriate 
10 parties. The transfer of responsibility for deliverables in step 543 generally includes 

M= the transition of training materials, operations manuals, and other supporting 

JS documents. 

O Continuing with the completion of the project in step 540, the organization 

yi 

%j evaluates the project, in step 544, by assessing the success of the project, summarizing 

15 the project's accomplishments, discussing/documenting any items for improvement, 

and channeling the resulting information through the appropriate quality management 

gasa 

ly process. The various results of the evaluation of the projects in step 544 should be 

;;i recorded in a closing memo, as specified in the CMML The results of the evaluation 

□ may include (1) reviewing the project work plan ; (2) updating the estimates; 

20 (3) sending the project's actual results to the owners of the estimating tool; 

(4) submitting final project metrics to the Software Engineering Process Group 
(SEPG); and (5) conducting an SQA debriefing to discuss results of the SQA program 
and also process improvement points. Another step in the completion of the project, 
step 540, is to release resources, step 545. The organization performs step 545, for 
25 instance, to "roll off human resources from the project and to return equipment and 

supplies to the appropriate custodian, thereby freeing these resources for use on other 
projects. 

Returning to FIG. 5A, the next task in the project management 500 is software 
quality assurance (SQA) review execution, step 550, the substeps of which are 
30 illustrated in FIG. 5F. During the SQA review execution of step 550, the organization 

may conduct software process and work product quality assurance reviews to verify 
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project adherence to standards and procedures. The first step of the SQA review 
execution 550 is to complete a project plan and metrics workbook, step 55 1. In this 
way, the project manager and SEPG liaison are encouraged and required to identify 
deliverables and processes to be reviewed; ensure that deliverables in the Project Plan 
and Work Plan are consistent; identify reviewers, reviewees, and review criteria; 
identify roles and responsibilities; identify SQA metrics; complete the quality 
program section of the project plan; and update the metrics workbook with the SQA 
review schedule. 

Another step of the SQA review execution 550 is to prepare for an SQA 
review, step 552. In the SQA review, the project manager provides job accounting 
information to the SQA reviewer and sets SQA review expectations. In preparation 
for the SQA review during step 552, a deliverable owner (i.e., a party responsible for 
producing a deliverable product or service) provides the deliverable to be reviewed, 
where "deliverable" is defined in the CMMI. The deliverable owner further provides 
contact and availability information to the SQA reviewer and provides review criteria 
and standards to the SQA reviewer. In response, the SQA reviewer gathers the 
deliverable product or service, reviews the proposed review criteria and standards, 
schedules a meeting with the deliverable owner, and receives job accounting 
information from the project manager. 

Returning to FIG. 5F, another step of the SQA review execution 550 is to 
conduct the SQA review, step 553. During the SQA review in step 553, the SQA 
reviewer generally reviews deliverables against review criteria/standards, identifies 
nonconformance items, and follows up with the deliverable owner as needed to meet 
the requirements of the CMMI. At the same time, a SEPG liaison reviews the project 
management deliverables against a best practices matrix. For the CMMI, the 
deliverable owner should be able to continue answering any questions. 

Another step of the SQA review execution 550 illustrated in FIG. 5F is to 
prepare the SQA report, step 554. The SQA reviewer prepares a detailed summary of 
findings and recommendations, including item number, date reported, and an accurate 
description of nonconformance items. The SQA reviewer then distributes the SQA 
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report to the deliverable owner and the SEPG liaison. The deliverable owner should 
then document responses in the SQA report template. 

Continuing with FIG. 5F, another step of the SQA review execution 550 is to 
discuss nonconformance items, step 555. Specifically, the organization should 
require the deliverable owner to discuss any nonconformance items with the SQA 
reviewer. In addition, the deliverable owner updates the SQA Report with proposed 
resolution(s) and projected completion date(s) for agreed upon items. The deliverable 
owner also escalates disagreement items for facilitation and updates a return report, as 
well as any necessary documents to the SQA reviewer for verification. In response, 
the SQA reviewer should discuss nonconformance items with the deliverable owner 
and verify the resolution of nonconformance items. During step 555, the SEPG 
liaison and the project management should also resolve escalated nonconformance 
items and resolve, on a case-by-case basis, any issues that may arise due to scheduling 
conflicts between the SQA reviewers and the deliverable owners. 

Continuing with FIG. 5F, another step of the SQA review execution 550 is to 
track SQA metrics, step 556. In step 556, the SQA reviewer sends the final report to 
the deliverable owner and the SEPG liaison. At this point, the SEPG liaison may 
update an SQA tracking tool and forward the final report and metrics to the project 
sponsor and project manager. Typically, the SEPG liaison includes metrics such as 
the SQA schedule variance; the number of nonconformance items; the cost/savings of 
the SQA program; the value added by conducting SQA reviews; and a best practices 
percentage showing compliance with desired best practice policies. As required by 
the CMMI, the project manager keeps copies of documentation and reports. 

Another aspect in the project management 500 is supplier agreement 
management, step 560, which is generally illustrated in FIG. 5G. The supplier 
agreement management 560 comprises subcontractor management in step 560(a) and 
product acquisition in step 560(b). Specifically, the subcontractor management in 
step 560(a) comprises the tasks of planning subcontractor management, step 561; 
organizing subcontractor management resources, step 562; controlling subcontractor 
management, step 563; and completing subcontractor management, step 564, as 
illustrated in FIG 5G. Likewise, product acquisition in step 560(b) comprises the 
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tasks of planning product acquisition, step 565; organizing product acquisition, 
step 566; controlling product acquisition, step 567; and completing product 
acquisition, step 568, as depicted in FIG 5G. 

FIG. 5H depicts that tasks 561(a)-561(f) comprise the planning of 
5 subcontractor management in step 561 . In step 561, project management plans for the 

project's use of subcontractors including developing criteria to be used for 
subcontractor selection. The first task in step 561 is to identify the need for a 
subcontractor, step 561(a). In step 561(a), the organization identifies a need for a 
subcontractor. Before the need for a subcontractor is determined, the business 
10 requirements for the project should be defined. The objective is to describe "what 

M= needs to be done and/or achieved" and which development team/s should be 

q instrumental in implementing this requirement. The supporting analysis and research 

;=r* provide input with regard to the requirements, including the current capability 

SJ analysis, constraint analysis, best practice research, and potential delivery options. If 

■ 15 the project team does not have the resources to satisfy these requirements, then a 

subcontractor should be considered. Again, the organization may use DAR if 
lU necessary to evaluate the need for a subcontractor. If a subcontractor is needed, the 

organization should update the supplier agreement management section of the project 
p plan with a description of the subcontractor arrangement. The organization may then 

20 prepare the subcontractor management plan. 

Returning to FIG. 5H, the organization's next action during the planning of 
subcontractor management in step 561 is to define a subcontractor statement of work 
(SOW), step 561(b). The subcontractor SOW should clearly define the scope and 
objectives of the subcontract, the process that should be used to manage the 
25 subcontractor, and any standard contract clauses. The SOW should also provide as 

much detail as possible about the planned subcontract, including the contract 
monitoring process, the quality management process, the configuration management 
process, and the contract closure process. A proposal/project team is generally 
responsible for identifying the technical requirements that the subcontractor should 
30 satisfy. 
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As depicted in FIG. 5H, the organization's next action during the planning of 
subcontractor management in step 561 is to develop subcontractor selection criteria, 
step 561(c). Prior to assessing subcontractors, the organization should define the 
selection criteria. Whereas some criteria should be generic, such as quality, service, 
value, and past performance, there is greater value in defining specific criteria that 
apply to different categories of assets and services to be procured, especially those 
criteria concerning longer-term cost considerations. Selection criteria should also 
reflect defined business needs. To satisfy this step 561(c), the proposal/project team 
should create the subcontractor selection criteria using the template provided. 

Next, in step 561(d), the organization should develop a subcontract pricing 
mode. In general, after defining the statement of work, it is necessary to establish the 
type of contract that will be used for the subcontract. It is important to determine the 
type of contract early in the process, as it has a fundamental impact on the 
subcontractor's proposal and economics of the program. This work should be closely 
coordinated with the development of the contract strategy. 

Returning to FIG. 5H, the organization's next action during the planning of 
subcontractor management in step 561 is to create a subcontractor long list, 
step 561(e). Using the subcontractor selection criteria template provided, the 
organization in step 561(e) identifies the long list of subcontractors that will be 
invited to propose. This list may be based on the following criteria: satisfaction with 
existing/previous subcontractor work, market share of the subcontractor, industry 
reputation of the subcontractor, proximity of the subcontractor, availability of the 
subcontractor, financial status of the subcontractor, etc. 

As depicted in FIG. 5H is the prepare/finalize request for proposal (RFP), 
step 561(f). The RFP should be created in step 561(f) after the need for a 
subcontractor has been established in step 561(a), the statement of work has been 
defined in step 561(b), the selection criteria have been established in step 561(c), the 
pricing model has been established in step 561(d), and the appropriate terms and 
conditions have been established. The RFP should be finalized with input from all 
relevant stakeholders. 
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As depicted in FIG. 5G, the next task in the supplier agreement management 
in step 560(a) is to organize subcontractor management resources, step 562. The 
organization performs step 562 to organize resources associated with subcontract 
management. In step 562, the project Work Plan is updated to account for 
subcontractors. Tasks that will use subcontractor resources are documented. 
Subcontractor Selection Criteria are finalized and a subcontractor is selected. Turning 
now to FIG. 51, the organization of subcontractor management resources in step 562 
comprises the tasks of developing work breakdown structure (WBS) and a resource- 
loaded work plan, step 562(a); finalize subcontractor selection criteria, step 562(b); 
issue a request for proposal (RFP) , step 562(c); receiving bids, step 562(d); 
evaluating bids to select a suitable subcontractor, step 562(e); and negotiating and 
finalizing a subcontract, step 562(f). It should be noted that steps 562(a)-(e) in the 
flow chart in FIG. 51 represent the potential tasks that would be completed to select a 
subcontractor, but many of these steps may be omitted based on project requirements. 

In the development of the WBS and resource-loaded work plan of step 562(a), 
the WBS decomposes each business capability into manageable units and depicts the 
total scope of the solution needed to achieve the program/project objectives. The 
work plan sets out the major work processes and constituent units of work that will be 
used to accomplish the project. The resource-loaded work plan then matches 
available resources with each task in the work plan. Both the WBS and the resource- 
loaded work plan should document the tasks that will be completed using 
subcontractor resources. 

In step 562(b), the organization should finalize subcontractor selection criteria, 
as depicted in FIG. 51. In step 562(b), the organization updates the subcontractor 
selection criteria established during the plan subcontractor management of step 561 to 
finalize the criteria that will be used to evaluate subcontractor proposals. Continuing 
with FIG. 51, during step 562(c), the organization next issues an RFP and distributes 
the RFP to a list of subcontractors identified for solicitation in step 561(e). The 
organization then receives bids, step 562(d), to gather proposals from subcontractors. 

The organization should then evaluate the bids and select a suitable 
subcontractor, step 562(e) in FIG. 51. In particular, as bids are received from 
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subcontractors, the responses should be entered into a subcontractor selection criteria 
matrix to facilitate the evaluation process. Evaluators should also review the potential 
risks associated with each subcontractor. Once all responses have been entered into 
the matrix and all potential risks have been assessed, a selection can be made by the 
5 organization. 

The organization should next negotiate and finalize a subcontract, step 562(f) 
in FIG. 51. After the subcontractor is selected, it may be necessary to make additional 
negotiations to finalize the contract. As a result of finalizing the subcontract, it may be 
necessary to update the project plan and/or subcontractor management plan with any 

10 new conditions, such as the need to provide project- furnished facilities. 

Returning to FIG. 5G, the next step in the subcontractor management in 
step 560(a) is to control subcontractor management in step 563. In step 563, the 
organization acts during project execution to monitor and control subcontractor 
activities for subcontractors that do not function as part of the project team. 

15 Subcontractors that work as part of the project team follow the processes outlined in 

the step of control project work in step 530. In addition, there should be regular status 
meetings with the subcontractor. During step 563, the work and work products of 
subcontractors are monitored through visual observation and/or Subcontractor Status 
Reports. Corrective action is taken as problems arise. 

20 Substeps 563(a)-(b) of the control subcontractor management in step 563 are 

depicted in FIG. 5J. Specifically, in step 563(a), the organization monitor 
subcontractor performance: The project manager or designated team member 
overseeing the subcontractor should observe the subcontractor's performance on a 
regular basis and manage all communications with the subcontractor. If the 

25 subcontractor fails to perform as expected (e.g., late delivery, poor quality, etc.), the 

organization should act to remedy these failures to minimize their harmful effects on 
the project. 

Likewise, in step 563(b), the organization should receive subcontractor 
reports, as illustrated in FIG. 5 J. The subcontractor should submit all reports to the 
30 project team as specified in the subcontract. This may include status reports, turn- 

around documents, invoices, metrics, etc. These reports should be used to track 
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subcontractor performance against the work plan and schedule milestones and 
evaluate quality of work. 

Returning to FIG. 5G, the final step in subcontractor management in 
step 560(a) is to complete subcontractor management, step 564. In step 564, project 
management verifies that the subcontractor has completed all tasks outlined in the 
subcontract and that technical performance requirements are satisfied. If the 
subcontractor successfully satisfies all contract requirements, both administrative and 
technical, the contract close out process occurs. If not, project management takes 
corrective action. Project Management updates the Closing Memo based on 
subcontractor deliverables and performance as necessary. As depicted in FIG. 5K, the 
tasks in the completion of subcontractor management in step 564 include the 
determination of whether contract requirements are satisfied, step 564(a); determining 
if technical performance requirements are satisfied, step 564(b); transitioning 
responsibilities and work products, step 564(c); and closing contract, step 564(d). In 
determining if contract requirements are satisfied in step 564(a), the organization 
assesses whether the subcontractor has failed to satisfy the contractual requirements. 
The organization further determines if any corrective actions may be needed. 

Continuing with FIG. 5K, in determining if technical performance 
requirements are satisfied in step 564(b), the project manager or designated team 
member oversees a subcontractor and is responsible for assessing the technical 
performance of that subcontractor. The acceptance criteria for contractual closeout 
are documented in the SOW and should be used to evaluate the subcontractor's 
performance. This assessment may include a review of deliverables, metrics, 
invoices, etc., submitted by the subcontractor. If the subcontractor fails to satisfy the 
technical performance requirements of the contract, corrective action may be needed. 

Referring again to FIG. 5K, the next task in completing the subcontractor 
management is the transition of responsibilities and work products, step 564(c): Once 
the subcontractor has successfully completed all work stated in the contract, it is 
necessary to transition the responsibilities and work products of the subcontractor to 
the appropriate party. Step 564(c) may require the subcontractor to train personnel in 
a given area, hand over system documentation and manuals, etc. 
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Then, in step 564(d), the organization may close the contract with the 
subcontractor, as illustrated in FIG. 5K. If the subcontractor successfully satisfies 
both administrative and technical contract requirements, the contract closeout process 
can occur. The contract closeout process may include the collection of information, 
5 such as performance metrics, from the subcontractor if this requirement was specified 

in the statement of work, request for proposal, or contract. 

Returning to FIG. 5G, the corollary to the subcontractor management of 
step 560(a) is the product acquisition of step 560(b). The first task in the product 
acquisition is to plan the product acquisition in step 565. The organization performs 

10 step 565 to plan activities related to product selection and implementation. In step 

565, the project's product needs are identified. The project's detailed approach to 
product acquisition is outlined in the Product Selection Approach. After determining 
a need for a product exists, a high-level review of the market is conducted to 
determine possible vendors and the product selection criteria are developed. It should 

15 also be noted that, in some cases, there are outside factors that govern the selection of 

products. Therefore, the following tasks 565(a)-565(d) may not always be necessary 
or inclusive. In addition, these tasks are only necessary when the product will be 
turned over to the client. 

Turning now to FIG. 5L, the first task in the planning of product acquisition in 

20 step 565 is to identify a need for a product, step 565(a). In step 565(a), the 

organization determines if business needs can be satisfied with the implementation of 
an off-the-shelf product. Step 565(a) may also involve participation from the 
proposal/project team and the client. The organization may generally follow the 
guidelines in a DAR Reference document. Specifically, the triggers and thresholds 

25 documented in the project plan determine if it is appropriate to use DAR to evaluate 

the need and/or selection of an off-the-shelf product. The organization may likewise 
use the project plan to describe the project's need for an off-the-shelf product and to 
identify the areas in which it may be necessary or desirable to use an off-the-shelf 
product. 

30 As depicted in FIG. 5L, the next task in the planning product acquisition in 

step 565 is to develop a product selection approach, step 565(b). In step 565(b), the 
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organization may document the project's detailed approach for product acquisition. 
Likewise, another task in the product acquisition 560(b) is to survey potential product 
providers, step 565(c). In step 565(c), the organization may conduct a high-level 
review of the market to determine potential product providers or contact an alliances 
group for assistance in identifying providers. The organization may then document 
potential providers on a vendor long list according to product selection criteria. This 
step 565(c) may include input from several different sources such as Internet research, 
participants with past project experience, independent product ratings, etc. In some 
cases, step 565(c) may not apply as the project sponsor may dictate the specific 
application to be used. 

Continuing with FIG. 5L, the next task in the planning of product acquisition 
in step 565 is to finalize the list the product providers to be invited to propose, 
step 565(d). In step 565(d), the organization identifies a list of product providers to 
be considered based on the information gathered during the survey of potential 
candidates. The organization may select providers that satisfy most of the project's 
requirements. In step 565(d), the organization may refer to predetermined product 
selection criteria with the product providers to be considered. 

Continuing with FIG. 5G, the next task in the product acquisition step 560(b) 
is to organize product acquisition, step 566. In step 566, the organization organizes 
resources associated with product acquisition. Furthermore, the product selection 
criteria are finalized and vendors are invited to demonstrate their products. At the end 
of this task, final product selections are made. It should be noted that, in some cases, 
there are outside factors that govern the selection of products, and, therefore, some or 
all of steps 565(a)-(e), described below in FIG. 5M, may be skipped as necessary. 

Turning now to FIG. 5M, the individual tasks that comprise the organization 
of the product acquisition in step 566. The first task is to finalize product selection 
criteria, step 566(a). In step 566(a), the organization should develop finalized 
selection criteria based on the organization's economic needs and goals, such as costs, 
timeframe, and quality concerns. 

Next, in step 566(b), the organization may define business scenarios, as 
illustrated in FIG. 5M. Using the product selection criteria formed in FIG. 566(a), the 
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organization may develop business scenarios that may then be used to form a 
questionnaire. In this way, the business scenarios may be used to evaluate product fit 
and performance during vendor demonstrations. 

The next task in FIG. 5M is to prepare and distribute a request for proposal 
5 (RFP), step 566(c). In step 566(c), the organization should develop a RFP that 

requires the vendors and providers to propose similar configurations and have all 
hardware, software, and on-site consulting services (in days) identified and itemized. 
Providers can also submit their idea of an optimal configuration. Furthermore, the 
RFP should include as much information about application interaction as possible. 
10 Returning to FIG. 5M, another task in the organization of product acquisition 

M in step 566 is to conduct vendor demonstrations, step 566(d). In step 566(d), each 

;5 finalist should provide a product demonstration. During the demonstration, the 

O organization should evaluate how well each provider/vendor meets the various 

y t 

Sf business scenarios. 

1 5 Next, the organization may compare costs and benefits of product providers, 

step 566(e), as illustrated in FIG. 5M. In particular, the organization may use the 
m product selection criteria to compare the proposals submitted by the product provider 

;=S| 

!j; finalists. Also evaluate the potential risks associated with each product. 

O Continuing with FIG. 5M, another step in the organization of the product 

20 acquisition is to make a final product selection, step 566(f). The organization may 

select the product provider based on the final scores in the product selection criteria 
and an assessment of potential risks. Step 566(f) may also includes the preparation of 
a purchase agreement or contract. It may also be necessary in step 566(f) to update 
the project plan to document any new conditions that result from the product 
25 acquisition, such as the need to provide project-furnished facilities. 

Returning to FIG. 5G, the next step in the product acquisition of step 560(b) is 
to control product acquisition, step 567. In step 567, the selected product is installed, 
testing is performed, and the performance of the product is evaluated. The 
organization may perform step 567 after acquiring the product to ensure that the 
30 product satisfies business needs and performs as anticipated. It should be noted that 

these tasks are typically performed only for new products that have not been 
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previously tested and implemented. For products that have been previously 
implemented, application testing and performance are evaluated during previous 
product and acceptance testing. 

The substeps in the control of product acquisition in step 567 are depicted in 
5 FIG. 5N. The first task in controlling product acquisition in step 567 is to install or 

otherwise use the product in the environment to be used for acceptance and 
performance testing, step 567(a). 

Next, in step 567(b), the organization conducts application testing, as 
illustrated in FIG. 5N. Specifically, once the product has been delivered, it is 

10 preferable in step 567(b) to perform a fit analysis to ensure that the software satisfies 

the business scenarios as originally intended. The fit analysis should map the product 
characteristics against both the existing user service class characteristics and the 
existing underlying components of the delivery vehicles (execution, operations, 
development, computing platforms, and network architecture). 

15 Continuing with FIG. 5N, the next task in the control of the product 

acquisition is to evaluate application performance, step 567(c). Three different 
methods are available to evaluate product performance in step 567(c): comparison, 
application sizing, and electronic spreadsheet analysis. Comparison analysis may be 
performed using existing installations of the product with similar environments, 

20 operations, and configurations. Some product vendors perform application sizing to 

determine if the product is adequate for the project needs, but results should be 
interpreted with caution. Electronic spreadsheet analysis translates business 
transactions into resource utilization and service times for evaluation. The accuracy 
of electronic spreadsheet analysis is driven primarily by the knowledge of business 

25 functions, transaction rates, and package architecture. 

Returning to FIG. 5G, another step in the product acquisition is to complete 
the product acquisition, step 568, to close out the product acquisition tasks. In step 
568, project management determines if the contract requirements are satisfied. Once 
the product has been assessed and meets all performance and functional needs, the 

30 product and the job responsibilities associated with the product are transitioned to the 

appropriate party. At this point, the contract with the vendor is closed. The tasks in 
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the completion of the product acquisition in step 568 are illustrated in FIG. 50. 
Specifically, the completion of the product acquisition in step 568 comprises the tasks 
of determining if contract requirements are satisfied, step 568(a); determining if 
performance issues require contract to be closed step 568(b); transitioning the 
acquired product, step 568(c); and closing the product acquisition contract, 
step 568(d). In the determination of whether purchase contract requirements are 
satisfied during step 568(a), the organization assesses the product against the contract 
requirements to determine if the agreed upon conditions have been met. Next, in 
step 568(b), the organization determines whether performance issues require the 
product purchase contract to be closed. Specifically, the organization decides if the 
product meets performance requirements. If the product fails to meet performance 
requirements, it may be necessary to terminate the contract. Contact the alliances 
group for assistance in identifying a product with better fit or performance. 

Once the product has been assessed and meets all performance and functional 
needs, it is necessary to transition the product and the job responsibilities associated 
with the product to the appropriate party. Accordingly, in step 568(c), the 
organization may transition the product as needed, as illustrated in FIG. 50. 
Step 568(c) may require the organization to train the appropriate party, handing over 
system documentation and manuals, etc. Then in step 568(d), the organization may 
close the purchase contract after verifying that contract requirements have been 
satisfied. 

Delivery Management 

R etttrntng to FTG. 1 ■ ■ ■■ th g-n ext step of the CMM in a BOX methodJ^ Ooft^^^ 
present invention is to implement delivery management 600. De^erjnlianagement 
600 relates to the activities undertaken to develo^^syst^msoftware application for 
eventual delivery to clients. The Delivei^management step 600 translates the 
required business outcomes-ifito a business solution. A business solution is the 
combination of businessprocess, a technology solution and organizational changes 
that collectively create value by improving business performance. The Delivery 
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d elivery pr o gr ams sh'»nM be > M^H fe^ 4 for ^ a ch business ^nlntj^ to he - deliv^rfiH - r 

In analysis stage 700, the organization accesses and defines the tasks to be 
accomplished for delivery of the desired products. During stage 700, business, user 
and interface requirements are defined as necessary to define and commit to a specific 
implementation and release plan. The information gathered during stage 700 focuses 
on business requirements, describing it to the level of detail needed to finalize the 
delivery releases, define the specific requirements, and resolve implementation issues. 
As illustrated in FIG. 7A, the analysis stage 700 consists of defining a business case, 
step 710; gathering and analysis of requirements, step 720; assessment of the 
deployment environment, step 730; and identification and analysis of the application 
and interface requirements, step 740. 

The first step in the analysis stage 700 is the defining of a business case, 
step 710, which is illustrated in FIG. 7B. In step 710, the organization defines the 
business case to determine benefits to be derived from, and justification for a 
proposed business solution. When defining a business case in step 710, the 
organization first determines an economic evaluation approach, step 711. 
Specifically, the organization performs this task to obtain commitments from the 
appropriate stakeholders in the sponsoring organization on the overall implementation 
approach for the proposed solution. This task treats the process of implementing a 
solution as an investment. 

The organization subsequently creates a model structure, step 712. During 
this task, the organization obtains an agreement from the sponsoring organization 
regarding the structure of the model used to determine the benefits of implementing 
the proposed solution. For example, benefits to be derived may be expressed in terms 
of increased market share or reduced operating costs. 

The organization next forecasts baseline business performance, step 713, to 
determine how the business should perform without the proposed solution. The next 
step in the analysis stage 710 is to project net change journey benefits, step 714, 
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during which the organization attempts to predict and quantify the benefits that the 
sponsoring organization should derive from implementing the proposed solution. A 
further step in the analysis stage 710 is to assemble a business case, step 715. During 
step 715, the organization documents a rationale for implementing the proposed 
solution. Ultimately, this document should serve as a motivational tool for change. 

Turning now to FIG. 7C, the next step in the analysis stage 700 is the 
gathering and analysis of requirements, step 720. In step 720, the current business 
environment is assessed and new requirements for the business and users are defined. 
During the gathering and analysis of requirements in step 720, the organization 
analyzes its current business, step 722, to obtain an accurate picture of its elements, its 
operation, and its performance. The organization then identifies user and business 
requirements, step 724, to define and document high-level requirements for desired 
solutions. These requirements include changes to human performance, business 
process, and technology. The organization should seek to ensure that these 
requirements meet the needs stated in the proposal, business needs statement, work 
request, or task order, including interfaces to other systems. During step 724, project 
participants impacted by the requirements should be involved in the review and sign- 
off of the requirements. Another step in the gathering and analysis of requirements in 
step 720 is to document the new business environment, step 726. In step 726, the 
organization documents user locations and transaction volumes in any new business 
environment. 

As illustrated in FIG. 7D, the analysis stage 700 continues with the assessment 
of the deployment environment, step 730, to ensure that deployment concerns and 
needs are considered sufficiently early in the development process. The objectives of 
the task are to consider the geographical, infrastructure, operational, and cultural 
differences between the current structure of the sponsoring organization and the 
desired structure, to define the deployment requirements, and to determine the optimal 
deployment mechanism. 

In the evaluation of the deployment environment in step 730, the organization 
assesses its release approach, step 732, to review the deployment plan, particularly to 
identify risks and to justify costs for deployment. The organization further identifies 



-65- 



Atty. Dckt. N^^3 




769-006 



deployment requirements, step 734, to identify deployment requirements for the 
proposed solution. A key deployment requirement is the production and release of the 
deliverable product or service. The organization should document the identified 
deployment requirements within a business requirements document. 
5 The next step in the analysis stage 700 is the identification and analysis of the 

application and interface requirements, step 740. During step 740, the application and 
interface requirements are prepared based on the business and user requirements 
gathered. All agreed-upon requirements gathered to this point are entered in the 
Requirements Traceability Matrix. Step 740 is generally illustrated in FIG. 7E and 
10 comprises these steps: transforming business requirements into more detailed 

application and interface requirements, step 741; integration of performance support 
requirements, step 742; recovering current application and interface design, step 743; 
identifying application and interface quality requirements, step 744; analyzing 
SI application and interface requirements, step 745; and verifying requirements 

|fl 15 documentation, step 746. 

During the transforming of business requirements into more detailed 
ilJ application and interface requirements in step 741, the organization uses the business 

Cj requirements as a starting point to develop the application requirements. The 

j~ application requirements should be in the context and scope of the business 

fissr. 

20 requirements. Also, these requirements should be verified to help ensure that the 

business process designs were properly interpreted. Then, in step 742, the integration 
of performance support requirements, the organization analyzes the tasks and factors 
that hinder user performance, taking into account their background and behavior. 

As illustrated in FIG. 7E, the next task in the identification and analysis of the 

25 application and interface requirements, step 740, is the recovery of current application 

and interface design, step 743. The recovery of current application and interface 
design in step 743 entails reviewing the current application/interface documentation 
and physical structures to gather requirements that may be omitted from the new 
application/interface. Step 743 includes the documentation of the present logical data 

30 structures. The organization should further identify expected requirements that 

otherwise may be assumed by the business representatives and not considered. 
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Another task in step 743 is to verify that the review also covers interface 
requirements. In this way, the recovery of current application/interface design in 
step 743 provides an inventory for conversion and a potential starting point for 
bottom-up data modeling. 

Subsequently, in step 744, the organization identifies application and interface 
quality requirements, as illustrated in FIG. 7E. During step 744, the organization 
seeks to select the quality attributes used to measure the application/interface 
functional and usability requirements, as these quality attributes should guide the 
design. Using these requirements, the organization should analyze application and 
interface requirements, step 745. Specifically, the organization should perform an 
analysis of the gathered requirements using process, event, data and content modeling 
techniques. Similarly, the organization may use validation techniques to confirm 
requirements such as prototyping and simulations. The organization may also create 
cases or scenarios to ensure requirements will be operational. The organization may 
additionally perform risk assessment against the identified requirements. The 
organization next documents the application and interface requirement specifications 
using a template. The actual requirements should be documented using a 
requirements traceability matrix for future tracking against other work products. The 
organization should make verify requirements are documented in a manner to ensure 
bidirectional traceability so that it is possible to trace requirements from the 
requirements development phase to the testing phase and vice versa. In addition, it 
should be possible to trace requirements across interfaces. In performing step 745, 
the organization preferably involves project participants impacted by the requirements 
in the review and sign-off of the requirements 

Returning to FIG. 7E, the next step is to verify the documentation of 
requirements, step 746. Specifically, the organization should review all requirement 
documents, such as executive architecture, development architecture, and operational 
architecture, thereby ensuring that these documents are in sync. 

Returning to FIG. 6A, the next set of tasks in the delivery management 600 is 
to design, step 800. The design stage 800 focuses on designing the components of the 
technology infrastructure, including the execution/operations and development 
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architectures. In addition, the design of the network, communication and computing 
platforms is performed in this stage. Design work should be coordinated with the 
development of the business processes, technical solution and organizational changes 
required to support the new infrastructure. The design process 800 is comprised of 
two tasks: designing the technology infrastructure, step 801 and designing the 
application, step 802. 

FIG. 8A illustrates one embodiment of the design of the technology 
infrastructure in step 801. One of the tasks in step 801 is to identify and analyze 
technology infrastructure requirements, step 810. During step 810, the organization 
prepares for the selection and design of the technology infrastructure and establishes 
preliminary plans for technology infrastructure releases and product testing. 
Furthermore, technology-related requirements are refined to form the component 
requirements for the technology infrastructure. For instance, step 810, the 
requirements for the technology infrastructure are outlined and preliminary plans for 
technology infrastructure releases and product testing are established. As this task is 
performed, technology-related requirements are refined to form the component 
requirements for the technology infrastructure. Accordingly, a first task in the 
identification and analysis of technology infrastructure requirements during step 810 
is to identify technology infrastructure requirements, step 81 1, as illustrated in 
FIG. 8B. The organization performs step 81 1 to identify the functional, technical, and 
performance requirements for the technology infrastructure that should support the 
solution. During step 81 1, the organization also identifies key performance 
indicators, creates baseline estimates of transaction volumes and system size, and sets 
measurable targets for the performance indicators. Key performance indicators 
examined during step 811 include resource availability, capacity, throughput, 
reliability, scalability, and usability. 

As indicated in FIG. 8B, a second process in the identification and analysis of 
technology infrastructure requirements in step 810 is to assess the technology 
infrastructure's current environment, step 812. In step 812, the organization assesses 
the ability of the existing technology infrastructure to support identified technology 
infrastructure requirements. 
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As depicted in FIG. 8B, the organization subsequently analyzes any potential 
technology infrastructure requirements, step 813, to refine the detailed functional, 
technical, and performance requirements for the technology infrastructure as outlined 
in the physical and performance models and to cover any additional requirements 
during the assessment of the current environment. The additional requirements may 
include user and service level requirements, as well as any requirements for the 
development architecture or the execution/operations architecture. The organization 
seeks to analyze and document the requirements for each component of the 
technology infrastructure and define additional needs. As part of step 813, the 
organization also seeks to involve all project participants impacted by the 
requirements in the review and sign-off of the requirements. 

Returning to FIG. 8B, other steps in the identification and analysis of 
technology infrastructure requirements in step 810 are (1) verification that 
requirements documentation is in sync, step 814, and (2) performance of risk 
assessment against the technical requirements, step 815. 

As illustrated in FIG. 8A, the next step in the design of the technology 
infrastructure, step 801, is the selection and design of execution/operation hardware, 
step 820. The organization performs step 820 to create and document high-level 
design and component design for the execution/operation architecture. Preferably, to 
prepare for testing of the architectural components, an architecture test plan, 
conditions, scripts and other needed family are also be created or defined during 
step 820. 

FIG. 8C depicts the individual steps of the selection and design of 
execution/operation hardware in step 820. A first step in the selection and design of 
execution/operation hardware in step 820 is to identify execution/operation 
architecture component options, step 821, so that the organization may create a list of 
suitable options for selecting and designing execution/operation architecture 
components that satisfy the technology infrastructure requirements. The organization 
then selects any reused execution/operation architecture components, step 822, if the 
execution architecture should utilize reused components from other projects, so that 
the organization may create a list of suitable options for selecting and designing those 
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components that satisfy the execution/operation technology infrastructure 
requirements. The organization may also select packaged execution/operation 
architecture components, step 823, if packaged components should be used in the 
project. The organization may perform step 823 to evaluate packaged products then 
and to gain the sponsoring organization's approval to continue. If suitable reusable or 
packaged components cannot be found, the organization may also choose to design 
custom execution/operation architecture components, step 824. If custom 
execution/operation components will be created in the project, the organization may 
then compare reused or packaged execution/operation solutions against custom- 
designed alternatives. 

Another step in the selection and design of execution hardware 820 is to 
design and validate the execution/operation architecture, step 825, to develop a 
complete design for the execution/operation architecture design after individual 
components have been selected or designed. The design for execution/operation 
architecture should also include custom component designs and any reused and 
packaged execution/operation architecture extension designs. 

Another step in the selection and design of execution/operation hardware 820 
is to develop an execution/operation architecture test plan, step 826, after the 
execution/operation architecture design is understood and documented, including the 
selection of reused and packaged execution/operation components. The primary goal 
of step 826 is to document test approaches and plans for the execution/operation 
architecture at the component and assembly level. 

The next step in the design of the technology infrastructure during step 801 is 
to select and design development architecture, step 830, as illustrated in FIGS. 8A and 
8D. The organization may perform this task to create and document the design of the 
development architecture components and test plans for those components. 
Specifically, the organization may create a high-level development architecture and 
component designs. Preferably, to prepare for testing of the architectural components, 
an architecture test plan, conditions, scripts and other needed family are also be 
created or defined during step 830. 
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FIG. 8D illustrates the substeps in the selection and design of development 
architecture in step 830. A first substep is to identify development architecture 
component options, step 831. In step 831, the organization may create and document 
the design of the development architecture components, as well as the test plan for 
those development architecture components. The organization also finalizes the 
physical model and selects or designs for development architecture components. 

Other tasks in step 830 include the selection of reused development 
architecture components from the existing technology infrastructure or from external 
sources, step 832, and the selection of packaged development architecture 
components, step 833, if they should be used in the project. If the organization should 
use any packaged development architecture components, the organization should 
determine which pieces of the development architecture to use and to negotiate 
contracts with vendors. In a preferred implementation of step 833, the organization 
also gathers additional information during vendor demonstrations and site visits to 
evaluate the available packaged development architecture components. 

Another substep in the selection and design of development architecture of 
step 830 is to design custom development architecture components, step 834, if any 
custom-designed components are needed. The organization may choose to produce a 
design for each custom component in order to understand the complexity, effort, and 
skills required to design and build the components efficiently. 

In another embodiment of step 830, the organization also designs and validates 
the development architecture, step 835, to review the development architecture 
requirements such as interfaces between components, to design custom development 
architecture components, and to incorporate any reused or packaged components. 
The organization may also develop a development architecture test plan, step 836. 
The organization should develop a test approach and a plan for testing, concurrently 
with the design and prototyping of the development architecture. Before developing 
the test approaches and plans for each component and the assembly of the 
development architecture, the organization should further review the objectives and 
scope for the component, component acceptance, and assembly test approach as 
defined in the test strategy. 
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^ptiirnina^l^— RA^a preferred embodiment ofjt hejjeUverv-maira^^ 
also includes a peer review, step 840, of the other steps 810-830 imdertakeii^ttriiTg" 
the process of designing the technology infrastrudi«^rstep^0o7 In the peer review, 
the organization verifies the acciy^e^and completeness of a deliverable product, 
whether it is a documen^efcode, for any step in the delivery stage 600. It should be 
appreciated th^l<^fiile displayed at this point in the CMM in a BOX method 10, a 
peer rejji^w 240 may be implemented at any time as necessary to satisfy the 

fements ol the CMM or CMM1 as well as oTheroverriding business concerns. 
Referring to FIG. 8E, the organization implements the peer review by first 
10 preparing for the review, step 842. Specifically, the project manager and team leader 

should budget time to conduct peer reviews and to establish peer review standards and 
criteria. Also, the owner of the deliverable product should identify and contact any 
peer review participants, schedule the peer review session, and distribute materials 
and standards to the reviewers. The reviewers should then prepare for the review by 
15 reading the materials prior to the peer review session and documenting comments and 

recommendations. Where appropriate, a peer review checklist may be used when 
conducting the peer review. 

Continuing with FIG. 8E, the next step in the peer review 840 is to conduct the 
peer review, step 844. During the peer review session in step 844, the deliverable 
20 owner should document any defects, issues, risks, and action items. The deliverable 

owner should also record meeting minutes and the time spent on the review. The 
reviewers are generally responsible for facilitating the discussion, sharing comments 
and recommendations with the deliverable owner, confirming that all issues are 
documented, providing metrics data, and scheduling a follow-up session if necessary. 
25 Next, in step 846, the organization should perform any necessary rework of 

the product, as depicted in FIG. 8E. During the rework in step 846, the deliverable 
owner implements the actions recommended by the reviewers, collects metrics data 
(including time spent preparing for review, number of defects found, etc.), and 
monitors the status of defects, issues, risks, and action items. As necessary, the peer 
30 reviewer should also verify that all pertinent items have been closed. 
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The organization should then analyze the review results, step 848 as depicted 
in FIG. 8E. The team leader submits the peer review metrics to the project manager 
for review. The project manager is then generally responsible for analyzing the 
metrics, evaluating the execution of the peer review process, and identifying areas for 
process improvement or corrective action with the peer review process. 

Returning to FIG. 6A, the next step in the delivery management, in step 600, is 
to design an application, step 802. As illustrated in FIG. 8F, during the design of the 
application in step 802, the organization designs an application architecture, step 850, 
to develop and document the conceptual and general design of the application and 
designs a database, step 860, to transform the data model into logical and physical 
designs of the application's database, while ensuring that data requirements should be 
met, and that data should be available through a conversion process. The design of 
the application in step 802 also entails planning a testing approach, step 870, for 
developing a comprehensive testing approach that should be used at all levels of 
testing, including component, assembly, product, user acceptance testing, and 
production readiness, i.e., deployment testing. Then, in step 880, the organization 
designs a performance support approach to determine existing workforce training 
needs, as well as to design methods and standards for performance support products to 
meet those workforce training needs. 

During the design of the application architecture in step 850, the organization 
seeks to develop and document the conceptual, general, and interface designs of the 
application. Preferably, to prepare for testing of the architectural components, an 
architecture test plan, conditions, scripts and other needed family are also be created 
or defined. As illustrated in FIG. 8G, the first step in the design of the application 
architecture in step 850 is to define the conceptual design, step 851. Specifically, the 
organization should document the operational concept for the solution in the 
conceptual design document. This documentation should outline the functional 
architecture of the proposed solution. 

Continuing with FIG. 8G, the organization should next determine whether to 
buy or build components, step 852, by reviewing the conceptual design and assessing 
factors such as historical information, corporate strategy, support infrastructure, 
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product availability, deadlines, and criticality of requirements. At this point, the 
organization should define an application architecture, step 853. When defining the 
application architecture in step 853, the organization should determine an approach 
for conducting design, such as calling group meetings for creating a conceptual 
approach. The organization should then evolve the conceptual design into a more 
detailed design as necessary for implementation with application. While evolving the 
design, key design decisions may trigger the need for a DAR, as described above. 

Continuing with the design of the application architecture in step 850, as 
depicted in FIG. 8G, the organization next undertakes the concurrent tasks of defining 
a process flow in step 854, designing application interfaces in step 855, and planning 
an assembly test in step 856. In defining a processing flow in step 854, the 
organization identifies all programs in the application, identifies their sequence, 
decomposes the programs into modules, and identifies how the modules 
communicate. The aim of step 854 is to develop enough detail to estimate the 
application's costs, resource consumption, and response times. 

At the same time, the organization designs application interfaces, step 855. 
Specifically, the organization designs the automated interfaces between the 
application being built and other applications with which it should communicate. 
During step 855, the organization also preferably develops an interface agreement and 
interface design to outline the expectations of the parties developing the various 
components. These documents should describe the handling of change requests, data 
exchange and control, backup and recovery requirements, error handling procedures, 
and provide escalation procedures in the event of a conflict. 

At the same time, the organization also plans assembly tests, step 856, by 
developing an approach and a plan that should be used to organize and execute 
assembly tests. The objective of assembly testing is to ensure that related components 
function properly when assembled into dialogs or batch strings and to verify that the 
component interfaces have appropriately implemented the design. 

As illustrated in FIG. 8F, the next step in the application design 802 is to 
design a database, step 860. When designing a database in step 860, the organization 
transforms the data model into logical and physical designs of the application's 
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database, acts to ensure that all data requirements should be met, and that all data 
should be available through the conversion process. The steps in the database design 
860 are illustrated in FIG. 8H. The first step in the database design 860 is to design a 
logical database, step 862. The organization may perform this task to transform the 
data model into the logical data structures using known database techniques. If the 
design of the logical database in step 862 produces a relational database, the logical 
database includes tables that contains various data used to define the database such as 
columns, primary keys, and row lengths; codes tables; foreign keys; integrity rules; 
views; and denormalization of the statistical data contained in the database. The 
logical data model is typically delivered to a client in soft copy format using data 
modeling tools. 

Next, the organization designs a physical database, step 864, by selecting or 
preparing physical storage and access structures for the application's data and by 
transforming the logical database design into storage and access structures that can be 
physically implemented. The physical database produced in step 864 generally 
includes database definitions, database space worksheets, database mappings, 
relational index definitions, and table space definitions. The database design 860 
continues with designing data conversion processes, step 866, such that the required 
conversion programs and procedures ensure the availability of data required by the 
application in production. In this step, the organization should produce an approach 
for converting and mapping documents. 

Returning to FIG. 8F, the design of the application in step 802 continues with 
the development of a planning testing approach, step 870. This planning testing 
approach should be used at multiple levels of testing such as component, assembly, 
product, and user acceptance testing, and deployment testing. As illustrated in 
FIG. 81, the first step in the development of a planning testing approach in step 870 is 
to develop an overall testing approach by refining and documenting an overall 
approach for testing, step 872. In developing the overall test approach, the 
organization should plan for the testing of interfaces. The overall test approach 
produced in step 872 should include details on sequence testing and the testing 
environment and also preferably includes the documentation of the resulting detailed 
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testing procedures. The next two steps in producing a testing approach in step 870 are 
(1) to identify product test conditions, step 874, where the conditions are used to 
verify that solutions meets the requirements for the components being created; and (2) 
to develop product test cycles, step 876. 

Returning to FIG. 8F, the next step in the application design 802 is to design a 
performance support approach, step 880, to determine existing workforce training 
needs, as well as to design methods and standards for performance support products to 
meet these workforce training needs. In step 880, the organization also designs 
performance support test and evaluation approaches and completes a validation of the 
complete test and evaluation approach. With reference to FIG. 8 J, the first step in the 
design of a performance support approach is to determine performance support needs, 
step 881, to determine the workforce's current proficiency and performance levels. 
This information is used to assess the gaps between current and expected proficiency 
and performance levels, which, in turn, drive the design of the performance support 
approach. Next, the organization designs learning objectives and a curriculum plan 
necessary to close the proficiency and performance gaps in the organization's 
workforce, step 882. Another step of the design of a performance support approach is 
to design performance support products, step 883, to define the delivery methods and 
standards for performance support. These delivery methods may include instructor- 
led training, performance simulation, computer-based training, videos, workshops, job 
aids, on-line quick reference tools, and training databases. 

As illustrated in FIG. 8 J, the next step is to design a comprehensive approach 
for testing the performance support products with respect to achieving each product's 
learning objectives, step 884. In step 884, the organization generally defines an 
approach that includes the scope and objectives of the test, environment requirements, 
entry/exit criteria, metrics, and schedule. The organization then designs a 
comprehensive approach for evaluating the effect of the performance support products 
on the employees' competency proficiency levels and performance levels in specific 
areas, step 885. Any designed approach for performance support evaluation should 
include evaluation methods, proficiency metrics, and schedules. The design of the 
performance support approach in step 880 may also include the verification and 
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validation of the performance support approach and the curriculum plan with 
stakeholders and subject matter experts, step 886. The organization should also 
organize labor review sessions to determine how well the sessions fit together to 
support the training needs of the workforce. 

As illustrated in FIG. 6A the next step in the delivery management 600 is to 
build and test, step 900. The build and test step 900 concentrates on implementing the 
business solution elements required for a single release. The delivering teams are 
responsible for the detailed design and creation of new processes, facilities, learning 
systems, performance support, application systems and technology infrastructure 
necessary to implement the new solution. These elements are then tested and 
implemented within a pilot environment. Thus, the building and testing in step 900 is 
accomplished through building and testing the technology infrastructure in step 901, 
building and testing the application in step 902, and planning executing product and 
acceptance tests in step 903. 

FIG. 9A presents the elements in the building and testing of the technology 
infrastructure in step 901. Step 901 focuses on acquiring, developing and testing the 
technology infrastructure. During step 901, additions and extensions to the 
execution/operations and development architectures are implemented, physical 
network and computing resources are developed, and a unified product is tested prior 
to the application product test. The first task in step 901 is to acquire physical 
environment assets and services, step 905. Generally, these physical environment 
assets and services are deployed to enable the implementation of the requirements 
based on the previously defined details of the physical environment assets. For 
instance, the organization may apply the data obtained in step 420. The organization 
uses the listing of physical environment assets and services to decide who should 
supply the assets and services, how the assets and services should be supplied, and 
how much the assets and services may cost. 

As depicted in FIG. 9B, the first step for acquiring physical environment 
assets and services is to initiate the acquisition of physical environment assets and 
services by selecting and appointing providers of assets and services, step 906. For 
instance, the organization preferably identifies those contracts that need to be 
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negotiated on an expedited basis and ensures that due diligence is applied to the 
context and content of all contractual arrangements. The organization then selects 
and appoints assets and services vendors, step 907, to appoint third-party suppliers 
and contractors who may provide assets, such as property and equipment, and 
technical/build/transfer/install/maintenance services for deployment of the physical 
environment, or services relating to decommissioning and disposal of the existing 
physical environment. 

Again, the organizations should prioritize those early purchase requirements 
that need to be expedited on a "fast track" basis. Subsequently, the organization 
should evaluate the deployment implications of the vendor appointments, step 908, to 
analyze the impact and deployment implications of appointing specific providers, 
either external or internal. These impacts may involve additions or revisions to project 
documents such as deployment plans, Business Case, project plan, and all subordinate 
plans. 

Returning to FIG. 9A, the next step in the building and testing of the 
technology infrastructure, step 901, is to build and test the execution/operation 
architecture, step 910, in order to complete a detailed design of the 
execution/operation architecture and to build and test that architecture. The 
organization may use the same methodology for application and operation 
development, as provided above in step 820, to plan and perform the component tests 
of the execution/operation architecture. As illustrated in FIG. 9C, the first step in the 
building and testing of the execution/operation architecture is to develop program 
specifications for each custom component of the execution architecture and to 
determine software configurations for each packaged or reusable component of that 
architecture, step 911. The organization may use the resulting detailed design to build 
custom components and to install packaged or reusable components. This task may 
also include updating the technology infrastructure component test plans, conducting 
reviews of the resulting detailed designs, and preparing common test data. 

The organization next builds any custom execution/operation architecture 
components needed for the project, step 912. This step 912 may also include 
documenting development procedures and standards, and conducting code reviews. 
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The organization then prepares and executes a component test of the 
execution/operation architecture components, step 913, to verify that the 
execution/operation architecture components are built according to proper designs. 
During step 913, any detected errors should be documented, and all of the execution 
architecture components should be relatively free of errors and ready for a subsequent 
assembly test. 

As depicted in FIG. 9D, the organization should similarly build and test the 
development architecture, step 915. The first task in step 915 is to perform a detailed 
design of the development architecture, step 916. In step 916, the organization 
develops program specifications for each custom component of the development 
architecture and to determine software configurations for each packaged or reused 
component of that architecture. Step 916 also preferably includes updating the 
technology infrastructure component test plans, conducting reviews of the resulting 
detailed designs, and preparing common test data. The organization should then build 
any needed custom development architecture components, step 917. Step 917 may 
also include documenting development procedures and standards, and conducting 
code reviews. The organization then prepares and executes a component test of the 
development architecture components, step 918, to verify that the development 
architecture custom components are built according to their designs. During step 918, 
the detected errors should be documented, and all of the development architecture 
custom components should be relatively free of errors and ready for the assembly test. 

As depicted in FIG. 6A, the build and test stage 900 also includes the building 
and testing of the application in step 902. Step 902 focuses on building and testing 
the application, creating training materials and other forms of performance support 
required by the business solution. During step 902, the detailed design, component 
testing and assembly testing of the application are completed. Learning products and 
business policies and procedures are developed to train and guide the users of the 
application. FIG. 9E illustrates the steps involved in the process to build and test the 
application, step 902. The first of these tasks is deployment planning, step 930, to 
produce deliverables that should be needed to test the application and interfaces in an 



-79- 



Atty. Dckt. N^p6769-006 

operations environment prior to deployment and to run the application and interfaces 
after deployment has occurred. 

Turning to FIG. 9F, the first task in the deployment planning during step 930 
is to develop a deployment approach to document the specifics of the major 
deployment activities, step 93 1 . The documentation should include items such as 
Data Conversion, Policy & Procedure Deployment, Risk Mitigation, Deployment 
Strategy, and workforce transition also should be covered in this document. The 
organization should next develop appropriate operating policies to produce a 
document outlining specific policies in the new operation environment, step 932. 
Responsibilities, system availability, and security should be documented in step 932, 
and upon completion of the project, this documentation should be given to the client. 
In a concurrent task, step 933, the organization may develop operating procedures by 
producing a document outlining the procedures that need to be followed during on- 
going support and operation of the installed application. Other subsequent tasks are to 
develop the operating organization to document the long-term organizational 
requirements that should be needed in the new operation environment, step 934, and 
to develop a disaster recovery plan that outlines an overall disaster recovery approach, 
as well as specific steps to follow during the disaster recovery process, step 935. The 
organization may then prepare the deployment test by creating a deployment test plan, 
test conditions, test scripts, and test data, step 936. This plan should be executed prior 
to delivery of a product to clients. Another step is to package operating manuals, so 
that the manuals may be turned over to client at completion of the project, step 937. 

The next step 940, the performing of application detailed design, is illustrated 
in FIG. 9G, and generally comprises a process to produce completed detailed design 
specifications that can be directly implemented in code, and a process to develop the 
approach and plan for component testing the application's modules. The first 
substep is to design and specify modules, step 941. Step 941 includes the production 
of a detailed design of the application and interfaces based on the general design and 
the application/interface requirements specification. During step 941, the 
organization should continue use of the chosen design methods to complete detailed 
designs. The organization should also prepare a detailed design of the application by 
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specifying all of the modules and their associated call patterns to the lowest level of 
detail. The detailed design of the application should also include describing each 
module's purpose and processing logic, developing database access patterns, and 
identifying other input/output operations. The organization should also be sure to 
address interfaces during the design process. In step 941, the organization should also 
update the interface agreement created during the design stage 600 to reflect any 
changes associated with the interfaces. The next task in performing a detailed design 
of the application in step 940 is to plan component testing, step 942, to verify the 
correctness of implementation of each of the application modules with respect to the 
application detailed design specifications. Step 942 includes determining common 
test data requirements and using the requirements to create common test data that can 
be used in the different stages of testing. 

As illustrated in FIG. 9H, the next step in the build and test stage 900 is to 
build and test the application, step 945. In step 945, the organization builds a 
complete, high-quality software application from the detailed design of the 
application. The organization may have developers implement the modules and then 
review the coded modules to verify correctness. The organization may also execute 
assembly tests to check interfaces and interdependencies between modules. One task 
in the building and testing of the application is the coding of modules, step 946, to 
create the code of each of the modules according to the previously created detailed 
application design specifications. Once the code is generated, the organization should 
check and compile the code as necessary for the project to identify and fix all errors, 
and to ensure that developers have followed any detailed coding procedures outlined 
in the project developer's guide. 

Continuing with FIG. 9H, the goal of the next step, the preparation and 
execution of the component tests in step 947, is to execute module code and verify 
that the module specification was correctly translated to the code. The module code 
should be verified using the component test conditions from the component test plan 
to prepare the test data and test scripts for the component tests. The organization 
should document and fix all detected errors before proceeding. The organization may 
then prepare and execute assembly tests, step 948, as needed to integrate modules and 
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verify that their interfaces and interdependencies are correctly designed and 
implemented. In step 948, the organization should use the assembly test conditions 
from the previously prepared assembly test plan to prepare test data and test scripts 
for the assembly tests. All detected errors should be fixed before proceeding. The 
next step, the development of a support program in step 949, involves coordinating 
and controlling the efforts of the development teams by supporting the programming 
and testing effort through supervision, control, and coordination. The organization 
may manage the programming and testing schedule, and monitor progress and report 
status, via the project management task packages outlined in the document repository 
policy defined in earlier steps. 

As depicted in FIG. 91, at this point in the build and test stage 900, the 
organization may develop a finalized, detailed set of policies and procedures, 
step 950. The business policies and procedures consist of rules governing work 
within the organization (policies) and the workflow for executing these rules 
(procedures). A first task in step 950 is to perform a detailed design of policies and 
procedures, step 952. In step 952, the organization should (1) define the product 
structure and design and (2) create and develop prototype templates for all policies 
and procedures. The organization should then develop business policies and 
procedures, step 954, by drafting a complete set of business policies and procedures to 
support the pending product release. In step 954, the business policies describe the 
business rules governing workflows and drive the development of business 
procedures and user procedures documentation. Similarly, the business procedures 
describe the sequential sets of tasks (and related resources, metrics, etc.) to follow 
based on the business policies. The organization should next validate and test these 
policies and procedures, step 956, to ensure that the Business Policies and Procedures 
meet the content of the requirements and can be executed by use of the applicable 
application. In step 956, the organization should further verify that the information 
collected is complete and accurately describes the processes. 

Turning to FIG. 9J, the next task in the building and testing of an application 
in step 900 is to develop learning products, step 960. In step 960, the organization 
selects the relevant authoring and development tools and to define standards, 
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templates, and development procedures. Step 960 further includes the defining of 
detailed learning objectives, determining learning context, and designing learning 
activities. The organization should also review paper-based learning product 
prototypes for ease of use. Also, the organization should develop activities and 
content, and define the support learners should require, and develop learning program 
evaluation materials for during-delivery and post-delivery evaluation of the learning 
process. Thus, in step 960, the organization should prepare and execute testing to 
ensure each learning product meets the stated objectives and instructors are effective 
when using the learning products. 

As depicted in FIG. 9 J, one task in the development of learning products in 
step 960 is to define learning product standards and a development environment, 
step 961, after the scope of the learning program has been defined and the learning 
requirements have been identified. In step 961, the organization should further select 
authoring and development tools and define the standards, templates, and procedures 
for the learning products. Development environments typically include Word or 
PowerPoint-based instructor-led materials or computer-based applications, but can 
also be made more robust with the use of job aids, a training database, on-line quick 
reference tools, and videos. 

Returning to FIG. 9J, concurrent steps in the developing of learning products 
in step 960 are (1) performing a detailed design of a learning program, step 962, to 
specify how each learning product identified in the learning product design should be 
built to meet the business needs of the organization and (2) prototyping learning 
products, step 963, to complete low-fidelity prototypes and conduct ease-of-use 
sessions on learning components (e.g., activities, support system, and instructor 
guide) of classroom-based learning products. 

In addition, the organization may create learning and evaluation products, step 
964, to develop the learning materials proposed and prototyped during the learning 
design activities. The creation of learning and evaluation products in step 964 
involves the developing of activities, content, and support materials that the learner 
will require to complete the learning product. Furthermore, evaluation tools are also 
preferably created in step 964 to ensure that learners have met the learning objectives. 
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Another possible task in the development of learning products in step 960 is the 
testing of learning products, step 965, which is best implemented after documenting 
participant profile, sample size, learning testing methods, test cycles, expected results, 
and script outlines. The goal is to test each learning product with the intended 
audience to ensure that the product meets the stated learning objectives, that the 
instructors are effective, and that the learning product meets the overall learning 
objectives for the release. The organization may also package the learning products, 
step 966, so that the learning products may be handed over to an appropriate 
stakeholder at the end of the project. 

At this point, the organization may plan and execute the product test and 
acceptance test, step 903, as illustrated in FIGS. 6 and 9K. Product tests evaluate 
whether the product is properly functioning, whereas acceptance tests evaluate 
whether the product functions as desired by customers. Step 903 focuses on 
performing a product test and user acceptance test on the new application to verify the 
application components and related technology, processes, and procedures work 
together properly according to the application and interface requirements. The first 
task in step 903 is to prepare and execute a product test plan, step 970, following the 
creation of the product test plan, conditions, scripts, and data that are used to execute 
the product test. The planning and execution of the product test plan in step 970 
should not begin until all requirements are finalized, the assembly test has been 
successfully completed, and the testing approach has been finished. 

As illustrated in FIG. 9L, to prepare and execute a product test, the 
organization should prepare a product test plan, step 971, to design and create the test 
conditions, test scripts, and test data for product testing. The organization should then 
review its product test plan, step 972, to verify that the product test plan created in 
step 971 is complete and accurate prior to product test execution. The resulting 
benefit to this check is that errors are caught early in the test process, where they can 
be addressed with minimal effort, rather than during test execution, where correction 
of errors becomes more costly. The organization should also create, cleanse, and 
convert data, step 973, to prepare the data for product test execution. If needed, the 
organization may confirm the product test environment, step 974, to verify that the 
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product test environment is ready for application product test execution by confirming 
that associated items are transferred to the test environment and that the identified 
configuration is complete and accurate. In this way, in step 974, the organization 
verifies that any tools needed for managing and executing the product test (for 
example, scripting tools and test data management tools) are installed and fully 
operational. This step 974 also helps ensure that the test data is properly copied and 
identifies responsibility and authority levels for managing code migration into the 
product test environment. 

Continuing with FIG. 9L, following confirmation of the test, the organization 
may execute the product test, step 975, to verify that the new application can work 
with the related technology, processes, and procedures to support the business 
processes successfully. The product test should prove: (1) that the new application 
and interfaces perform according to the application/interface requirements established 
in prior steps, and (2) that the application can operate effectively in concert with all 
other production applications and all available end-user manuals and procedures. 

If any problems arise during the testing, the organization may perform product 
test fixes, step 976, to analyze and resolve all problems identified during product test 
execution as illustrated in FIG. 9L. Typically, the organization assigns each problem 
to a specific team member for correction. After a problem is fixed, the organization 
may reexecute the test condition to verify that the fix was successful, and perform a 
regression test to ensure other components were not adversely affected by the fix. 
Once all errors have been resolved the product test can be considered complete. 

Returning to FIG. 9K, the organization may next prepare and execute 
acceptance tests, step 980. The organization performs step 980 to create the test plan, 
test conditions, test scripts, and test data for user acceptance testing. The user 
acceptance test (UAT) also validates that the solution supports the business 
performance model and should not begin until successful completion of the product 
test in step 970. The UAT verifies that the solution works according to the 
requirements and meets the business objectives. As depicted in FIG. 9M, steps 981- 
986, the preparation and execution of the acceptance tests during step 980 are very 
similar to steps 971-976. For instance, the initial step of the preparation and 
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execution of the acceptance tests in step 980 is to prepare a user acceptance test plan, 
step 981, including plans for testing interfaces and the application. The organization 
then reviews the user acceptance test plan, step 982, to ensure that the user acceptance 
test plan is complete. The next step is to create, cleanse, and convert data, step 983, 
as needed, to prepare the data required for the acceptance testing, including producing 
new data, converting existing data, and reconciling different data representations and 
different database schema representations. If necessary, the organizations may also 
confirm user acceptance of the test environment, step 984, to ensure: (1) that the user 
acceptance test environment is ready for test execution by checking that all necessary 
items are transferred to the test environment, (2) that the identified configuration is 
complete and accurate, and (3) that any tools required during the acceptance test are 
installed and fully operational. 

At this point the organization executes the user acceptance test, step 985, to 
test the interaction between the components of the solution to verify and validate that 
they support the model. This acceptance test helps to ensure that the solution works 
according to the requirements and meets the business objectives. If any problems 
arise in the test, the organization may resolve user acceptance test issues, step 986. 
Specifically, the organization may utilize the user acceptance test issues to analyze all 
problems identified by the user acceptance test execution through investigating each 
problem, and assigning it to the appropriate development team for correction. After a 
problem is fixed, the organization should reexecute the test condition to verify the fix 
was successful. The organization may also perform a regression test to ensure other 
components were not adversely affected by the fix. Once all errors have been 
resolved in step 986, the acceptance test may be considered complete. 

Once solutions to a problem have been analyzed in step 700, designed in 
step 800, and built and tested in step 900, an organization may deploy the complete 
solution, as depicted in FIG. 10A. The deployment stage 1000 is conducted to 
transition the organization to the new business solution. The deployment stage 1000 
includes the activities required to transform the personnel, business process, and 
technology elements required to establish the business solution. The deployment 
stage is repeated for each deployment site, which is the organizational or geographic 




-86- 



Atty. Dckt. N^p6769-006 

unit that will receive the business solution. The first step in the deployment is to 
transition users and to deploy policies and procedures, step 1010, to evaluate the 
existing workforce of an organization in terms of roles and skills, and perform a gap 
analysis against the new organization infrastructure for the deployment unit, as 
illustrated in FIG. 10B. In step 1010, the organization may finalize the workforce 
infrastructure, step 101 1, to mobilize the people who should eventually use the 
solution. At the same time, the organization should examine the organizational 
structure, as well as the skills and roles of the existing workforce, to determine if the 
resources needed to support the solution exist. If needed roles or skills are missing, 
another task in step 101 1 is to develop a plan to address the gaps. This task should be 
performed before selecting, hiring, or assigning people to teams. 

As illustrated in FIG. 10B, the next task is to redeploy the workforce, 
step 1012, to transfer existing users into the different roles, teams, or functional areas 
needed to support the solution. Concurrently, the organization recruits and selects a 
workforce, step 1013, after developing a profile of the combination of skills and other 
characteristics necessary to support the solution and using the resulting profile to 
select internal individuals and to hire external individuals who can fill the necessary 
roles and teams. The organization then trains the trainers, step 1014, by preparing the 
instructors and coaches who should eventually train the workforce to use the solution. 
Step 1014 generally entails conducting practice sessions of the course in order to 
allow instructors to rehearse their delivery with course developers as the audience. 
Next, the organization implements orientation and training, step 1015. Specifically, 
the organization introduces employees to the solution that should be deployed. To 
maximize the benefits of training in step 1015, the instructors should be trained in 
step 1014 prior to the training of the workforce. The organization may further give 
users information on the context of the solution within the organization and train them 
on how to operate the solution. Furthermore, the organization preferably identifies 
individual and team development needs, and workers should provide feedback on the 
learning program in order to improve the process for future releases. Step 1015 
should be performed after selecting and recruiting individuals to fill the roles and 
teams, and after developing the training materials and job aids. 
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Concurrent with above-described steps 101 1-1015, if needed in response to 
the deployment, the organization may install the new business policies and 
procedures, step 1016. In step 1016, the organization also acts to en sure that all 
pieces of the new business policies and procedures are available. 

The next step of deployment stage 1000 is to deploy the physical environment, 
step 1020, as illustrated in FIG. 10C. In step 1020, the organization manages the 
implementation of changes to facilities, equipment, and other physical assets. Upon 
completion of step 1020, a formal exchange of the transformed physical environment 
from the project team to the sponsor's operating management may occur. 

Continuing with FIG. 10C, one task in deploying physical environment in step 
1020 is to initiate physical environment deployment, step 1022, to mobilize the 
internal and external resources to prepare the physical environment for the solution 
that should be deployed, and to establish the necessary communication channels. One 
aspect of step 1022 is to verify that all of the involved parties understand what work 
for which they are responsible, when this work is scheduled, and how this work is 
interdependent with the tasks assigned to others. Other tasks in step 1022 may include 
defining how to monitor, expedite, and report progress. The organization may 
optionally determine how to maintain quality control and how to regularly 
communicate progress with stakeholders. Also, step 1022 may include planning for 
formal progress and quality control reviews. 

Returning to FIG. 10C, the next step in the deployment of the physical 
environment during step 1020 to is to manage physical environment transformation, 
step 1024, to carry out the development and configuration of the physical 
environment needed to support the solution. The management of physical 
environment transformation in step 1024 includes expediting progress, managing 
issues and risks that may impact the implementation plan, and providing management 
with summary progress reports. 

Continuing with FIG. 10C, another the next step in the deployment of the 
physical environment during step 1020 is to complete a physical environment 
handover, step 1026. During 1026, the organization acts to ensure that the 
development and configuration of the physical environments are complete, and are 
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transferred to, and accepted by, the sponsoring organization's operations 
management. Step 1026 generally occurs when both the stakeholders and the 
deployment project management team are satisfied that the implementation has been 
completed successfully. 

As depicted in FIG. 10D, the next task is to deploy the application, step 1030, 
to transition the new application and its operating environment into the deployment 
unit. During step 1030, the organization may establish the data required by the new 
application; configure the operating environment to the needs of the deployment unit; 
install the application; configure application parameters needed for the deployment 
unit; and verify that the application is correct and consistent for the deployment. 
Tasks in step 1030 may include the creation, cleansing, and conversion of data, 
step 1032, as needed, to establish the data to be used with the new application. 
During step 1032, an organization may produce new data and reconcile different data 
representations and different database schema representations. The organization may 
also convert an existing electronic representation of data into a format to be used by 
the new application or use a data conversion application to convert data from an 
existing database to the new database. 

As depicted in FIG. 10D, a concurrent task is to configure the application, step 
1034 in order to configure and customize the new application and the existing 
operating environment to the needs of the deployment unit. Next, the organization 
installs the application, step 1036. Specifically, the organization may, during step 
1036, install and customize the application components of the business capability in 
the deployment unit, making sure that all pieces of the new application are available. 
Another task in the deployment of the application during step 1030 is to verify 
application, step 1038, by installing and customizing the new application components 
of the business capability in the deployment unit, making sure that all pieces of the 
new application are available. 

As illustrated in FIG. 10E, another step in the deployment stage 1000 is to 
deploy the technology infrastructure, step 1040. During step 1040, the organization 
preferably outlines of the procedures and considerations for deploying technology 
infrastructure components at a deployment unit. Likewise, the organization should 
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address the potential differences in technology infrastructure environments between 
deployment units. The goal of step 1040 is to bring the deployment unit up to the 
technology infrastructure baseline required for the business capability. Deployment 
of the technology infrastructure in step 1040 may also include the commissioning and 
decommissioning of infrastructure components. To deploy the technology 
infrastructure in step 1040, the organization may also configure the technology 
infrastructure, step 1042, to customize the deployment unit's technology 
infrastructure in preparation for the new business capability components. Step 1042 
generally does not handle the configurations that are part of the installation of any 
new technology infrastructure elements. Next, the organization installs the 
technology infrastructure, step 1044, to install the technology infrastructure 
components of the business capability. The organization should also verify the 
available technology infrastructure, step 1046, so that whenever a technology 
infrastructure component is added or modified, the organization performs this task to 
verify the new technology infrastructure environment and addresses the discoveries of 
the testing. This verification in step 1046 is generally completed only for the 
technology infrastructure. 

The next task in the deployment stage 1000 is to activate and test a solution, 
step 1050, to verify the deployment and launch the new operating management 
processes. Step 1050 generally includes actions required to finalize performance 
targets, to remove redundant legacy elements, and to stabilize the deployment unit for 
transition to operations management. One task in the step 1050 is to verify workforce 
and business readiness, step 1051, after successful completion of the acceptance test 
and after all elements have been deployed, but before the business capability is 
activated. Step 1051 includes execution of the deployment test and verifies that the 
workforce and the other elements of the business are prepared for operation on the 
first day and all subsequent days. The organization may use the SIRs or CRs to 
record any errors encountered. 

A concurrent task is to verify team and process readiness, step 1052, after all 
elements have been deployed, but before the business capability is activated. Step 
1052 verifies that the deployment team and the deployment processes are prepared to 
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deployment, step 1053, to activate and verify the capabilities that have been deployed. 
In step 1053, any of the organization's various teams should have the confidence and 
ability needed to proceed with irreversible decisions, such as the removal of legacy 
systems and procedures. The organization should now begin to operate the deployed 
business capabilities. 

Next, the organization may remove legacy elements, step 1054, to remove the 
legacy systems from old operations and management processes after making the 
irreversible decision to proceed with the new business solution. Concurrent with step 
1054, the organization should finalize performance targets, step 1055 to formalize the 
baseline for continuous improvement of the business solution. The finalizing of 
performance targets is initiated as soon as the business solution has been operating 
long enough to collect reliable data for adjusting the business performance model. 

In another step, the organization may deploy stabilization, step 1056, to 
prepare the transition of business capabilities to operations management. The 
organization should also monitor the progress over a period of time to verify the 
stability of the team using the deployed business capabilities. A decision that the 
product is ready to release is reached by analyzing the actual performance and 
productivity forecasts of the team using the deployed business capabilities. 

Turning now to FIG. 6B, maintenance, step 610, is the continuing support of 
an application, addressing both production problem resolution (through SIRs) and 
application enhancements (through CRs). The first task in the maintenance is to 
review the SIRs or CRs, step 611. With a SIR, repair work needs to be completed 
immediately, whereas a CR may be incorporated into a subsequent release of the 
application. The organization may also review incident or change requests for risk as 
well. Another step in the maintenance 610 is to perform an impact assessment, 
step 612. Specific activities in step 612 include investigating the SIR; determining the 
change required to address the identified problems to resolve the SIR; determining the 
effort involved; developing alternatives; and selecting the acceptable alternative. Any 
affected work products altered by the SIR such as requirements, designs, work plans, 
code, etc., should be updated as necessary. If it is determined that no application 
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change is needed, the system should be retested to ensure that the problem no longer 
exists or that the problem should be forwarded to the appropriate channels. 

Another task in the maintenance 610 is to design application changes, step 
613, to create the application design that is needed to build the solution. The 
organization may also build and test application changes, step 614, to perform the 
work necessary to implement the desired change. Once the change has been 
completed, the change should be component tested and product tested to ensure that it 
is working properly. Additionally, a regression test should be performed in step 614 
to help ensure that other peripheral functions were not affected by the change. Next, 
the organization may roll out changes, step 615, as needed to implement the designed, 
developed, tested changes into the production environment. 

For CRs corresponding with desired enhancements to the product, the 
organization may also follow the program delivery life cycle, step 616: For changes 
(CRs) that can be incorporated into a scheduled release, the detailed work involved in 
modifying the existing application is performed according to the task packages/tasks 
in the delivering phase 600, including the analysis, design, build and test, and 
deployment steps 700, 800, 900 and 1000. In this way enhancement that extend 
beyond the original scope of the product are developed much like a new product. 



System 

^These-sldlledjbl-fo^ 

embodiments of the CMM iiLa-BOX-method 10 descnbecfabove may be implemented 
in vai ious v t or instance, the organization may use a set of written templates 

dpe gfing the4mpleiiieuUtliuii uf th e - tasks in thfXMM in a RO X ^nethod 10: 

J n one imple mentati on, the pre s e nfc4nventionr4na^b e implement e d - a s a ^ 

computer application that prompts an organizationjoi^xa^ its 
operation and structure. Usingtii£S©4flpTifCthe application then creates a series of 
task lists to implemgnHfi^MM in a BOX method 10 of the present invention. The 
applicatipn-tnay further create a record of task lists, so that the organization may 
j>a<rf y daerarusiii4 te-a etkms a* required in rt re-ffofM and i;mivil Alternatively, tfre- 
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program may P"™ *g -te™p1fl tfi S thmugb^whtch H i p n r gaiw/ntinn miy Hnrnroe n t i K 1 

^activitiesT 

Id particular-th o s e s killed in the art -wtfrr ecogiiize that v arious-c mbodimcn is^ 
of the CMM in a BOX method 10 described above may be implemenleeHIsing a 
combination of both electronic hardware and software. Refejriilg to Fig. 1 1 A, a 
CMM implementation system 1 100 receives user inmit^Tl30 and produces a business 
organization plan 1 140 based on the user inpjjH130. The system 1 100 may be, for 
example, a personal computer (PC), asctfver, or any other computer device used for 
such purposes. The system 1 lpOinay be coupled to a database 1 120 containing 
information on the organisation and its suppliers. In this embodiment, the system 
1 100 has an organisation management module 1 1 10, a program management module 
1 1 12, a proj^t management module 1114 and a delivery management module 1116 
for implementing organization management 100, program management 400, project 
pwiage ment . 500, and jdelw^y- man a gcment 600. - 

If-the-Gon^uteF-de-vice-LLOO-is^for exampleT-a-netwoFk-serverj-in-eieGtromc 
communication with an electronic network, then users 1 160 mayJ>e~ableto use the 
CMM system 1 100 remotely. Referring to F^J-lB^Kowing the computer device of 
Fig. 1 1 A in electronic communicajiofrwith a network 1 150. The network 1 150, may 
be, for example, the InJjerflgCan intranet, an extranet, a Value Added Network 
("VAN"), Vnjuafl™duct Network (VPN) and the like. Users 1 160 may transmit user 
input^iafa 1 120 to the CMM system 1 100 via the electronic network 1 150 then obtain 
izafioirpta n 1 140 based on the input dat a 1 130 . — . 
rln-anotireremBMin^^ 
may be a soft^^i^appllSaHoirdesigned to operate over various hardware and 
5mgut^systo»isras4aiowrt in the art. ' 



Conclusion 

<-The GMM-methocLof-the prpsentAtLventiftywha^rheen empirically shown fc£Z= 
allow organizations to achieve higheijevelsjof^^ much more rapidly. 

On average, an org^iiizatiSnOTa project within an organization takes about three 
Y par^tcrachieve complianc e with level 3 ^f th n GMM T" c ontrast; seve r al p r ojects - 
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knpfementmg the CMM in a BOX meth o d 10 of Hie preae ni invention have reached^ 
level 3 of the CMM in an aversg^effflfiTmonths. These results suggest the utility and 
benefit of thg^reSentinvention in assisting organizations to achieve higher levels of 

The foregoing description of the preferred embodiments of the invention has 
been presented for the purposes of illustration and description. It is not intended to be 
exhaustive or to limit the invention to the precise form disclosed. Many modifications 
and variations are possible in light of the above teaching. For instance, the method of 
the present invention may be modified as needed to meet the requirements of new 
versions of CMM and other maturity models as they are developed. It is intended that 
the scope of the invention be limited not by this detailed description, but rather by the 
claims appended hereto. The above specification, examples, and data provide a 
complete description of the manufacture and use of the composition of the invention. 
Since many embodiments of the invention can be made without departing from the 
spirit and scope of the invention, the invention resides in the claims hereinafter 
appended. 
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